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Abstract 

The  mission  of  an  aerobatic  team  is  to  enhance  government  recruiting,  and 
credibly  represent  an  Air  Force.  For  that  reason,  Turkish  Air  Force  Command  plans  to 
replace  the  current  aerobatic  aircraft,  which  have  completed  their  lifetime.  Before  making 
this  type  of  decision,  Operational  Research  Directorate  in  the  TurAF  Headquarters 
perfonns  different  research  to  construct  an  overall  decision  guide.  Although,  that 
department  constructed  very  successful  decision-making  guides,  the  analysts  need  a  more 
systematic  approach  for  these  types  of  complex  problems. 

The  purpose  of  this  research  is  to  lessen  the  potential  human  errors  by  using 
decision  analysis  and  cost  analysis  techniques.  This  research  uses  Value-  Focused 
Thinking  in  conjunction  with  Multi-Objective  Decision  Analysis  and  Cost  Analysis.  It 
creates  a  decision-making  model  that  allows  decision  makers  to  interact  with  analysts  by 
specifying  his/her  objectives,  values  and  preferences.  On  the  other  hand,  cost  analysis 
process  detennines  cost  effective  alternatives.  Alternatives  are  scored  using  17  evaluation 
measures  that  are  identified  by  interviews  with  the  decision  maker  and  stakeholders. 
Robustness  of  the  model  is  tested  by  sensitivity  analysis.  Results  of  sensitivity  analysis 
on  current  weights  indicate  that  this  model  has  robust  results.  Nevertheless,  sensitivity 
analysis  on  cost  indicates  that  many  issues  may  influence  the  robustness  of  cost  estimates 
and  the  best  cost  effective  alternative. 
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VALUE  FOCUSED  THINKING  IN  DEVELOPING  AEROBATIC  AIRCRAFT 


SELECTION  MODEL  FOR  TURKISH  AIR  FORCE 


I.  Introduction 


1.1  Background 

In  early  years  of  aviation  history,  there  were  different  reasons  to  use  flight 
maneuvers.  One  of  the  most  known  is  aerobatic  displays.  The  first  air  meet  that  included 
the  first  aerobatic  performance  was  in  1909.  By  development  of  aviation,  air  meets  and 
air  shows  became  very  popular  during  the  1920s.  In  the  time  up  to  World  War  II 
international  competitions  in  aerobatics  were  developed  (Hickoksports,  2011).  In  order 
to  increase  the  interest  of  the  air  force  among  young  people,  many  countries  formed 
aerobatic  demonstration  teams.  One  of  the  leading  countries  was  Turkey. 

Ataturk,  the  founder  of  Turkish  Republic,  recognized  the  importance  of  aviation 
to  national  security  in  1925,  saying,  “The  future  is  in  the  sky.  Nations  who  fail  to  protect 
their  skies  can  never  be  sure  of  their  tomorrows”.  The  Turkish  Air  Force  (TurAF)  is  one 
of  the  oldest  air  forces  and  has  operated  more  than  180  different  types  of  aircraft  since 
1911.  TurAF  currently  ranks  second  in  NATO  in  terms  of  fleet  size,  behind  only  the 
USAF.  Supported  by  long-range  in-flight  refueling  capability,  the  fighter  jets  of  the 
TurAF  participate  in  international  operations  and  exercises  on  all  continents  of  the  world. 

The  Air  Force  Command  has  the  purpose  of  “Preventing  threats  and  dangers 
likely  to  be  received  via  air  against  Turkish  country  and  Turkish  Nation,  and  facilitating 
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the  way  to  success  of  the  duties  of  Land  and  Naval  Forces  during  a  possible  war.” 
Another  important  duty  of  the  TurAF  is  promoting  itself  and  Turkish  Republic  both  at 
home  and  abroad  and  endearing  aviation  by  enhancing  relationships  with  the  public.  For 
that  reason,  the  first  official  aerobatics  demonstrations  started  in  1926  within  the  Air 
Force  School. 

After  becoming  a  NATO  member,  the  TurAF  modernization  period  accelerated 
after  1952.  In  TurAF  history,  the  first  aerobatic  display  teams,  corresponding  to 
technological  modern  world  standards,  was  formed  soon  after  the  introduction  of  the  jets 
in  1952.  By  changing  its  name  and  type  of  aircraft,  six  different  aerobatic  demonstration 
teams  were  established  in  different  years.  The  sixth  of  the  TurAF  aerobatic  display  teams 
formed  by  F-5A  (Freedom  Fighter)  aircraft  in  1971.  After  a  period  of  20  years,  the 
TurAF  decided  to  re-establish  its  aerobatics  activities.  For  that  purpose,  the  last  aerobatic 
demonstration  teams  of  the  TurAF  obtained  the  call  sign  of  Turkish  Stars  in  1992 
(TuRAF  100th  Anniversary,  2011). 

The  Turkish  Stars  are  the  aerobatic  demonstration  team  of  the  TurAF  and  the 
national  aerobatics  team  of  Turkey,  which  is  the  proof  of  capabilities  of  the  modem 
Turkish  Air  Force.  Since,  the  highest  level  of  skills  and  knowledge  is  necessary  in  order 
to  perform  air  shows  to  the  audience.  Turkish  Stars  fly  with  eight  Northrop  F-5  Freedom 
Fighters,  making  them  one  of  the  few  national  aerobatics  teams  to  fly  supersonic  aircraft, 
and  the  only  one  with  fonnations  of  eight  supersonic  jets.  Their  aim  is  to  “Present  the 
TurAF  on  a  national  and  international  platfonn,  to  represent  the  Turkish  Republic,  to 
contribute  to  the  efficiency  and  discipline  of  the  TurAF,  to  reinforce  the  peoples’  trust  in 
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the  Air  Force,  and  to  increase  the  interest  of  the  air  force  among  the  young  people  of 
Turkey  (Turkish  Stars,  2011). 

1.2  Problem  Identification 

In  order  to  accomplish  its  goals,  maintain  flight  safety  and  protect  its  reputation, 
the  TurAF  needs  to  keep  up  with  the  latest  aviation  technology.  For  that  reason,  the  Air 
Force  Command  plans  to  replace  the  F-5  aircraft  which  has  been  used  since  1971  and 
which  has  completed  their  lifetime.  A  decision  making  process  will  be  used  by  TurAF  to 
procure  twenty  aircraft  in  total. 

Before  making  decisions  for  big  projects,  the  Operational  Research  Directorate  in 
the  TurAF  Headquarters  perfonns  different  research  to  construct  an  overall  decision 
guide  for  decision  makers.  However,  this  process  is  performed  by  using  the  experience 
and  knowledge  of  the  analysts.  Although,  that  department  constructed  very  successful 
decision-making  guides,  the  analysts  need  a  more  systematic  approach  for  these  types  of 
complex  problems  and  project  decisions. 

1.3  Research  Questions 

In  order  to  construct  a  robust  decision-making  model,  this  study  asks  the  following 
research  questions: 

•  Is  it  possible  to  construct  a  robust,  objective  and  traceable  decision-making 
model,  which  lessens  human  errors,  reduces  the  amount  of  total  working  time  and 
selects  the  best  alternative  among  diverse  number  of  alternatives  with  different 
attributes? 
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•  Who  are  the  stakeholders  of  the  system  that  affect  the  decision  process  and  are 
affected  by  that  decision? 

•  What  are  the  desired  characteristics  of  an  aerobatic  aircraft  that  will  satisfy  the 
decision  maker’s  and  stakeholders’  demands? 

•  Is  it  possible  to  create  a  model  that  will  specify  the  most  cost  effective 
alternative? 

1.4  Research  Approach 

The  motivation  of  this  research  derives  from  the  multiobjective  decision  problem 
of  the  TurAF  aerobatic  aircraft  selection  problem.  This  research  presents  a  decision¬ 
making  methodology  for  top-level  decision  makers  who  make  strategic  decisions  about 
military  projects  and  system  selection. 

Decision-making  for  this  type  of  decisions  is  a  process  characterized  by 
complexity,  uncertainty,  multiple  objectives,  and  combination  of  different  data  types. 
Most  of  today’s  important  project  management  decisions  are  complex.  Without  proper 
analysis,  it  is  hard  to  make  choices  among  diverse  numbers  of  alternatives.  The 
complexity  of  decision-making  depends  on  many  reasons.  Most  problems  in  decision¬ 
making  involve  multiple  objectives  and  uncertainties.  The  number  of  alternatives  can  be 
significant  and  make  decision  making  more  complex.  Multiple  stakeholders  with 
different  objectives  and  preferences  make  that  process  harder.  Hence,  it  is  not  easy  to  see 
all  of  the  different  aspects  of  a  problem.  Keeney,  1992  describes  that  as  “In  an  uncertain 
world,  the  responsible  decision  maker  must  balance  judgments  about  uncertainties  with 
his  or  her  preferences  for  possible  consequences  or  outcomes.”  Decision  maker  may 
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make  poor  decisions  without  proper  approach.  In  addition,  overtime  these  poor  decisions 
become  a  burden,  especially  decisions  that  are  connected  to  large-scale  projects  that 
affect  many  people  and  consume  a  big  budget  (Virine  &  Trumper,  2008). 

Strategic  decisions  are  the  combination  of  high-level  uncertainty,  long-term 
consequences,  interaction  between  different  factors  that  cause  complexity,  and 
necessitation  of  identifying  key  stakeholders  to  evaluate  effects  of  that  decision  on  a 
system  (Zopounidis,  2010).  As  a  result,  decision-makers  face  great  distress  while  they  are 
making  strategic  decisions.  Therefore,  strategic  decision-making  needs  to  follow  a  well- 
organized  and  well-designed  systematic  process.  Decision  Analysis  (DA)  allows 
decision-makers  to  overcome  the  problems  associated  with  the  strategic  decisions  by 
using  analytical  results.  Hence,  it  mostly  eliminates  the  intuition  of  the  decision  maker. 
The  decision  analysis  process  entails  a  logical  analysis  of  a  correctly  structured  problem, 
identification  of  creative  alternatives  based  on  reliable  infonnation,  implementation  of 
the  selected  alternative  and  an  evaluation  of  the  results.  According  to  Keeney,  1992  “DA 
is  a  practical  framework  of  methods  and  tools  to  promote  creativity  and  help  people  make 
better  decisions.” 

The  decision  process  of  selecting  an  appropriate  aerobatic  aircraft  effects  many 
issues.  The  most  critical  one  is  the  fame  of  Turkish  Air  Force  and  Republic  of  Turkey. 
Since,  Turkish  Stars  perform  many  national  and  international  shows  all  around  the  world 
in  order  to  accomplish  its  goals.  Another  main  issue  of  any  poor  decision  that  has 
negative  consequence  is  the  cost  of  the  project.  Because  this  project  includes  a  minimum 
30  year  period,  present  cost  can  be  enormous  due  to  a  wrong  decision.  On  the  other  hand, 
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many  stakeholders  have  different  positions  that  they  face  as  a  result  of  different  aspects 
of  the  decision.  In  addition,  they  may  have  different  objectives  that  can  affect  the  final 
decision.  Hence,  the  problem  just  by  using  personal  intuitions  or  expert  opinions  will  be 
very  complex  to  make  a  decision.  Realizing  that  poor  decision  making  in  large-scale 
projects  can  result  in  high  costs  and  cause  harm,  governments  and  private  businesses 
recognize  the  importance  of  decision  analysis  techniques.  As  a  result,  decision  analysis 
process  is  essential  in  order  to  make  decisions  that  are  more  appropriate.  From  that  point 
of  view,  the  purpose  of  that  research  is  to  lessen  the  potential  human  errors  that  can  cause 
a  poor  decision  by  using  decision  analysis  and  cost  analysis  techniques. 

Complexity  and  interactions  between  different  factors  constitutes  conflicting  and 
multiple  objectives.  Once  a  decision  maker  defines  his  or  her  fundamental  objectives,  a 
technique  called  Value  Focused  Thinking  (VFT)  is  appropriate  for  decision-making 
process  and  evaluation  of  alternatives.  Using  VFT  for  decision-making  process  enables 
objective,  traceable  and  robust  results  (Jackson,  Jones,  &  Lehmkuhl,  1996).  Hence,  it 
leads  to  a  systematic  approach  for  decision  making  which  interacts  with  decision  maker 
and  stakeholders  by  minimizing  human  related  problems. 

Different  types  of  aircrafts  with  different  characteristics  may  be  challenging  to 
compare  and  decide  at  first  glance.  For  that  reason,  this  research  creates  a  decision- 
support  model  that  allows  decision  makers  to  interact  with  analysts  by  specifying  his/her 
objectives,  values  and  preferences.  On  the  other  hand,  the  cost  analysis  process 
determines  cost  effective  alternatives.  For  that  reason,  this  research  uses  Value-  Focused 
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Thinking  (VFT)  in  conjunction  with  Multi-Objective  Decision  Analysis  (MOD A)  and 
Cost  Analysis. 

1.5  Research  Assumption 

This  research  makes  the  following  assumptions: 

a.  Only  one  type  of  aircraft  will  be  the  final  decision  among  the  alternatives. 

b.  There  is  no  limitation  on  suppliers.  TurAF  may  purchase  any  aircraft  from  any 
supplier. 

c.  TurAF  has  a  technological  background  and  eligible  personnel  to  support  any  type 
of  aircraft. 

d.  Lifetime  of  each  aircraft  is  30  years. 

e.  Aircraft  features  remain  the  same  to  the  end  of  the  service  life. 

f.  Evaluation  measures  give  the  proper  weighting. 

g.  Weighting  accurately  depicts  the  decision  maker’s  preference. 

Section  4  includes  sensitivity  analysis  of  proper  weighting  assumption  (f)  and 
weighting  accuracy  assumption  (g).  Hence,  accuracy  of  assumptions  is  tested.  Other 
assumptions  are  inherent  from  the  problem.  Therefore,  there  is  no  sensitivity  analysis, 
which  tests  accuracy  of  other  assumptions. 
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II.  Literature  Review 


The  purpose  of  this  chapter  is  to  provide  a  brief  review  of  Turkish  acquisition  rules 
and  regulations  and  literature,  theory  and  researches  related  to  Decision  Analysis,  Value- 
Focused  Thinking  and  Life  Cycle  Cost  Analysis. 

2.1  Turkish  Procurement  Procedure 

Procurement  is  the  process  of  acquiring  goods  and  services  from  other 
organizations.  Its  purpose  is  to  obtain  cost,  time,  quality  and  performance  efficient  goods 
and  services.  Companies  show  great  deal  of  interest  to  obtain  the  optimal  efficiency  level. 
In  recent  decades,  government  organizations  became  a  part  of  that  struggle.  However, 
different  from  companies,  governmental  rules  and  organizations  control  governmental 
procurement  activities.  These  factors  create  a  great  amount  of  restriction  on  procurement 
processes. 

Defense  procurement  is  one  of  the  most  critical  governmental  procurement 
activities.  Since,  the  main  purpose  of  the  defense  procurement  is  to  maintain  modem  and 
combat  ready  armed  forces.  Turkish  Anned  Forces  procurement  activities  have  some 
governmental  regulations.  Following  sections  will  review  the  regulations  and  rules  that 
are  used  for  defense  procurements  for  Turkish  Anned  Forces  to  see  if  there  is  any  legal 
restriction  to  apply  VFT. 

2.1.1  Turkish  Public  Procurement  Law 

The  purpose  of  Turkish  Public  Procurement  Law  is  to  establish  principles  and 
procedures  that  apply  in  any  governmental  procurement.  According  to  the  Law,  the 
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definition  of  procurement  is  “  the  proceedings  which  involve  the  award  of  a  goods, 
services  or  works  contract  to  the  tenderer  selected  in  accordance  with  the  procedures  and 
conditions  laid  down  in  this  Law,  and  which  is  completed  by  signing  of  the  contract 
following  the  approval  of  the  contracting  officer”.  Energy,  water,  transportation  and 
telecommunication  sectors  are  out  of  the  scope  of  the  Law.  Also,  the  law  has  some 
exceptions  which  are  defined  in  article  3b  of  the  Public  Procurement  Law.  According  to 
article  3b  “Goods,  services  and  works  procurement  which  are  decided  by  the  relevant 
ministry  that  these  are  related  to  the  defense,  security  or  intelligence  or  that  these  require 
to  be  treated  confidentially,  or  procurements  requiring  special  security  measures  during  the 
perfonnance  of  the  contract  pursuant  to  related  legislation  or  those  concerning  the  cases  in 
which  the  basic  interests  of  the  state’s  security  needs  to  be  protected.”.  Depending  on  that 
article,  Turkish  Armed  Forces  use  2009/14973  enact  to  procure  goods,  services  or  works. 
2,1.2  2009/14973  Enact 

2009/14973  enact  establishes  the  principles  and  procedures  to  be  applied  on 
procurement  related  to  Turkish  Anned  Forces  modernization,  preservation  and 
development.  Activities  related  to  procurement  begin  with  Turkish  Anned  Forces 
demand  to  Defense  Industry  Counselor.  Prior  to  the  procurement  proceedings  of  goods, 
services  or  works,  the  entity  shall  conduct  all  necessary  price  research  and  shall 
detennine  an  estimated  cost  by  using  cost  analysis,  ft  is  not  desired  to  exceed  that 
calculated  estimated  cost.  Estimated  cost  is  not  being  stated  in  tender  or  pre-qualification 
advertisements.  Cost  Estimation  Calculation  Statement  describes  all  of  the  details  for 
estimating  cost. 
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Other  step  is  preparing  specifications  list.  Preparation  of  administrative  and 
technical  specifications  specifying  all  characteristics  of  the  goods,  services  and  works  that 
constitute  the  subject  matter  of  the  procurement  by  the  contracting  entities  is  essential  and 
mandatory.  Procured  subject  must  satisfy  these  specifications.  If  candidate  procurement 
subject  does  not  have  all  of  the  specifications,  negotiations  are  allowed  to  rearrange  the 
subject  and  the  price.  Three  kinds  of  documents  are  prepared  to  detennine  the 
specifications.  Administrative  Specifications  Document  specifies  procurement  objectives, 
tender  participations  requirement,  evaluation  criteria,  contracting  processes,  quality 
requirement  and  quality  assurance  issues.  Pre-qualification  Document  is  used  for  restricted 
method  of  contracting.  It  specifies  information  for  pre-qualification  application  and 
evaluation  factors.  Technical  Specification  Document  specifies  all  of  the  technical 
information.  That  includes  sampling  issues,  acceptance  issues,  handling  and  packing 
requirements  and  warranty  issues  (Turkish  Public  Procurement  Law,  2002). 

In  procurement  of  goods,  services  and  works  by  contracting  entities,  open 
procurement  procedure  or  restricted  procurement  procedure  may  apply.  Defense  Industry 
Counsellorship  decides  procurement  procedure.  Open  procedure  is  a  procedure  where  all 
tenderers  may  submit  their  tenders.  If  all  specifications  are  satisfied,  the  agreement  is 
made  following  the  price  negotiation.  The  lowest  bidder  who  meets  technical  and  financial 
specifications  wins  the  tender.  Restricted  procedure  is  a  procedure  in  which  tenders  who 
are  invited  following  pre-qualification  by  the  contracting  entity,  can  submit  their  tenders. 
Procurement  of  goods,  services  or  works  may  be  conducted  by  restricted  procedure  where 
open  procedure  is  not  applicable  as  the  nature  of  the  subject  necessitates  speciality  and/or 
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high  technology  and  in  procurement  of  works.  This  evaluation  is  based  on  the  Pre¬ 
qualification  Document.  If  all  specifications  are  satisfied,  the  agreement  is  made  following 
the  price  negotiation. 

Material  and  service  support  is  made  by  direct  procurement.  The  method  of  direct 
procurement  may  be  applied  in  international  agreements  like  MOA  (Memorandum  of 
Agreement).  Another  way  is  agreements  with  foreign  military  defence  ministries  like  FMS 
(Foreign  Military  Sales)  which  is  made  with  United  States  Military  Defence  Ministry. 
Agreements  with  military  organizations  like  NAMSA  (NATO  Maintenance  and  Supply 
Agency)  are  a  different  procedure  to  supply  material  and  service  support.  Agreements  can 
be  made  with  foreign  military  material  sales  organizations  that  are  under  government 
control  such  that  BWB  (Germany  Federal  Office  for  Defense  Technology  and 
Procurement  Company  Center)  (2009/14973  Enact).  For  domestic  defence  procurement, 
enact  is  used. 

2,1.3  Applicability  of  VFT 

Due  to  Turkish  Procurement  Law  and  2009/14973  enact,  there  is  no  restriction  to 
apply  VFT  for  the  aerobatic  aircraft  procurement  procedure.  In  fact,  Turkish  procurement 
procedures  utilize  Alternative  Focused  Thinking  (AFT).  It  is  obvious  that  direct 
procurement  and  open  procurement  types  specify  the  alternatives  first  and  then  construct 
all  analysis  depending  on  those  alternatives.  Restricted  procurement  type  is  better  than 
other  two  types  of  procurement,  but  it  needs  further  systematic  approach.  The  following 
section  describes  differences  between  AFT  and  VFT. 
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2.2  Decision  Analysis 

As  well  as  the  individual  decision  makers,  business  executives  and  governmental 
decision  makers  deal  with  hard  decision  problems.  Decision  problems  are  hard;  since, 
they  have  inherent  complexity  and  uncertainty  in  addition  to  multiple  and  conflicting 
objectives.  Also  because  of  the  different  perspectives,  a  decision  can  end  up  with  diverse 
numbers  of  conclusions  and  consequences.  Decision  analysis  can  help  to  deal  with 
complexity  by  providing  effective  methods  for  organizing  the  problem.  It  also  helps  to 
identify  the  source  of  uncertainty  and  represent  in  a  systematic  and  useful  way.  Decision 
analysis  provides  useful  tools  to  represent  and  deal  with  multiple  and  conflicting 
objectives.  It  helps  to  sort  and  resolve  different  perspectives  regardless  of  the  number  of 
decision  makers  and  stakeholders  (Clemen  &  Reilly,  2001). 

The  main  purpose  of  decision  analysis  is  to  help  decision  maker  to  make  decision 
about  specific  topics  by  combining  the  problem  heritage  with  his  or  her  preferences  and 
beliefs.  Instead  of  struggling  with  the  big  problem,  decision  analysis  decomposes  the 
problem  into  smaller  pieces  that  are  easy  to  understand  and  solve.  Decision  analysis  is  a 
guideline  for  systematic  thinking;  but  it  does  not  give  the  exact  solution  or  best  result  for 
decision  problems.  It  is  as  a  tool  that  gives  insights  about  trade-offs  and  reveals  a 
recommended  course  of  action  (Clemen  &  Reilly,  2001). 

The  first  step  and  the  most  essential  step  of  decision-making  is  to  identify 
decision  problem  and  decision  maker’s  objectives.  Without  clear  identification  of 
objectives,  it  is  hard  to  obtain  recommended  decision.  However,  in  many  cases  decision 
maker  specify  multiple  objectives.  Multiobjective  decision  analysis  (MODA)  provides 
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insight  to  decision  makers  faced  with  multiple  and  probably  conflicting  objectives  and 
significant  uncertainties.  Multiple  conflicting  objectives  reveal  tradeoffs  between  these 
objectives.  Since,  it  is  very  hard  to  satisfy  all  of  the  objectives  at  the  same  time.  Figure  1 
shows  overall  process  used  by  MODA. 


Figure  1  MODA  Process  (Feng  &  Keller,  2006) 

There  are  several  types  of  approaches  as  an  alternative  to  MODA.  Depending  on 
the  type  of  the  decision  problem,  different  types  of  methods  may  be  applied.  Multicriteria 
Decision  Analysis  (MCDA)  is  one  of  the  most  widely  used  approaches.  According  to 
Kirkwood,  1997,  “The  phrase  MCDA  refers  to  a  set  of  related  approaches  for  analyzing 
multiobjective  decisions  using  mathematical  programming  (optimization)  methods.” 
There  are  many  MCDA  methods  in  literature.  According  to  Chen  &  Hwnag,  1991, 
classification  of  MCDA  methods  depends  on  the  type  of  information  and  the  salient 
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features  of  the  information.  Figure  2  represents  that  classification.  According  to 
(Triantaphyllou,  2000)  the  most  commonly  used  methods  are  Weighted  Sum  Model 
(WSM),  Analytic  Hierarchy  Process  (AHP),  Weighted  Product  Model  (WPM), 
ELECTRE,  and  TOPSIS.  Yoon  &  Hwang,  1991  contains  detailed  overview  of  these 
techniques. 


Type  of  Information  Salient  Feature  Major  Classes 

From  the  DM  of  Information  of  Methods 


Dominance 


No  Information  Maximin 


Maximax 


Conjunctive  Method 
Disjunctive  Method 


Lexicographic  Method 
Lexicographic  Semi  Order 
Elimination  by  Aspect 

Weighted  Sum  Model 
Weighted  Product  Model 
Analytic  Hierarchy  Process 
ELECTRE 
TOPSIS 


Figure  2  Examples  MCDA  Methods  (Chen  &  Hwnag,  1991) 

Data  and  number  of  decision  makers  are  some  other  ways  to  categorize  MCDA 
(Triantaphyllou,  2000).  Over  recent  decades,  different  types  of  MCDA  techniques  have 
been  derived.  Figueria,  Greco  &  Ehrgott,  2005  includes  an  overview,  foundations, 
different  approaches,  applications  and  software  for  recent  MCDA  techniques. 

Contrary  to  MODA,  MCDA  methods  do  not  explicitly  asses  a  value  or  utility 
function.  They  commonly  use  mathematical  programming  techniques  to  identify 
recommended  alternatives  (efficient  set).  Most  of  the  MCDA  techniques  do  not  separate 
value  of  utility  function  from  the  ranking  of  alternatives.  Instead,  the  decision  maker 
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examines  various  alternatives  and  provides  information  that  a  mathematician  can  use  to 
propose  different  alternatives.  For  example,  AHP  method  does  not  require  identifying  a 
value  function.  It  requires  pair  wise  comparisons  of  alternatives.  Then,  a  mathematical 
method  is  required  to  estimate  value  function  based  on  the  various  pair  wise  comparison. 
If  the  number  of  alternatives  is  large,  then  this  method  requires  many  pair  wise 
comparison.  MCDA  methods  need  computer  software  or  complex  algorithms  to  make 
evaluations  and  to  give  insight  to  decision  maker  (Kirkwood,  1997). 

2.3  Creativity  in  Decision  Making 

In  real  world  decision  making  problems,  most  of  the  time  decision  makers 
generates  criteria  for  present  alternatives.  Hence,  they  stick  with  alternatives  and  do  not 
make  the  most  efficient  decision.  Creativity  is  an  excellent  basis  to  develop  new 
alternatives.  These  creative  alternatives  are  detennined  by  considering  objectives  of 
decision  maker.  According  to  Clemen  &  Reilly,  2001  “A  creative  alternative  has  both 
elements  of  novelty  and  effectiveness,  where  effectiveness  is  thought  of  a  decision  maker 
,  a  group  of  individuals,  or  even  the  diverse  objectives  held  by  different  stakeholders  in  a 
negotiation.”  If  the  decision  maker  efficiently  uses  these  creative  thinking  activities, 
he/she  may  convert  the  decision  problem  to  a  decision  opportunity  in  addition  to 
developing  creative  alternatives  (Keeney  R.  L.,  1992). 

There  are  several  creative  thinking  techniques.  One  of  them  is  Value-Focused 
Thinking  (VFT).  VFT  uses  fundamental  and  mean  objectives,  to  develop  decision 
alternatives.  Following  section  includes  detailed  clarification  of  VFT.  Fluent  and  Flexible 
Thinking  is  another  way  of  thinking  creatively.  Fluent  Thinking  necessitates  writing 
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many  new  ideas  quickly.  On  the  other  hand,  flexible  thinking  generates  fewer  ideas, 
which  includes  a  broader  area.  A  creative  thinking  process  may  use  these  techniques 
individually  or  together.  Idea  Checklist  develops  solutions  for  a  decision  problem  or  lists 
attributes  to  define  alternatives.  In  addition,  it  generates  strategies  by  using  strategy 
generation  tables.  The  most  known  technique  is  Brainstorming  which  requires  thinking 
activity  among  different  people.  Metaphorical  thinking,  Nominal  Group  Technique  are 
some  of  the  other  techniques  for  creative  decision  making  (Clemen  &  Reilly,  2001). 

2.4  Value  Focused  Thinking 

The  purpose  of  decisions  is  achieving  something,  which  is  defined  by  the  values 
for  given  decision.  Objectives  specify  those  values  in  detail.  VFT  keeps  attention 
throughout  the  decision  process  on  what  you  hope  to  achieve.  According  to  Kirkwood, 
“VFT  is  a  strategic,  quantitative  approach  to  decision-making  that  uses  specified 
objectives,  evaluation  measures,  and  value  hierarchies.”  VFT  utilizes  the  fundamental 
and  mean  objectives  to  specify  alternatives  and  decision  opportunities.  VFT  uses  MODA 
as  a  mathematical  technique  (Parnell,  2007).  VFT  is  a  MODA  process  that  reverses  AFT 
approach  by  first  defining  the  values  that  are  important  to  DM  and  stakeholders. 

Values  should  be  in  the  center  of  the  decision  making  processes.  Keeney  R.  L., 
1992,  as  shown  in  Figure  3,  identifies  the  central  role  of  thinking  about  values.  VFT 
allows  you  to  explicitly  state  DM’s  objectives,  uncover  hidden  objectives  and  clarify  the 
problem.  With  explicit  objectives  and  values,  it  is  possible  to  create  and  evaluate 
alternatives  appropriately.  After  identifying  values,  useful  information  can  be  realized 
which  would  be  lead  better  consequences.  Discussing  values  will  improve 
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communication  and  understanding.  By  using  values,  it  is  possible  to  manage  stakeholders 
who  must  be  the  part  of  decision  process  and  negotiations. 


Figure  3  Overview  of  Value-Focused  Thinking  (Keeney  R.  L.,  1992) 

2.4.1  Value  Focused  Thinking  and  Alternative  Focused  Thinking 

VFT  and  AFT  are  two  methods  that  apply  to  decision  problems.  According  to 
Keeney,  1992,  AFT  first  necessitates  figuring  out  available  alternatives  and  then 
choosing  the  best  one.  By  that  method,  analysis  focuses  on  the  alternatives  of  interest  to 
DM.  Yet,  sometimes  none  of  the  alternatives  satisfies  DM  objectives.  On  the  other  hand, 
DM  objectives  may  not  be  a  direct  part  to  evaluate  alternatives  (Parnell,  2007).  If 
available  alternatives  are  bad,  then  the  AFT  approach  will  only  chose  the  best  of  bad 
alternatives. 

VFT  consists  of  two  activities  first  deciding  what  is  desired  and  then  figuring  out 
the  ways  to  obtain  it.  VFT  starts  with  specifying  values  that  DM  and  stakeholders  define. 
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It  uses  values  to  generate  and  evaluate  alternatives.  Hence,  VFT  is  a  constraint-free 
thinking,  which  ends  up  with  an  efficient  decision(Bell,  Raiffa,  &  Tversky,  1988). 

2.4.2  Elements  of  Value  Focused  Thinking 

Before  applying  VFT,  we  need  to  understand  some  basic  concepts  that  will  allow 
thinking  systematically. 

Objectives 

According  to  Keeney  &  Raiffa,  1993  “An  objective  is  a  statement  of  something 
that  is  desired  to  be  achieved.  It  indicates  the  direction  in  which  we  should  strive  to  do 
better.”  Some  examples  are  “maximizing  profit”,  “maximize  safety”,  “minimize  damage” 
etc.  It  is  different  from  goal.  Since,  goal  is  the  specified  level  of  achievement  of  these 
objectives. 

Objectives  Hierarchy 

After  defining  multiple  objectives,  a  systematic  structure  called  objective 
hierarchy  (or  value  tree)  is  used.  At  the  higher  level  of  hierarchy,  objectives  are  more 
general.  However,  lower  levels  of  hierarchy  have  specific  objectives.  For  example, 
“maximize  transportation  safety”  is  at  the  higher  level,  “minimize  fatalities”  is  at  the 
lower  level  of  the  hierarchy.  Therefore,  we  can  define  lower-level  objectives  or  as  the 
branches  of  higher-level  objectives  (Bell,  Raiffa,  &  Tversky,  1988).  While  constructing 
hierarchy,  fundamental  objective  approach  and  means  objective  approach  can  be  used 
(Keeney  R.  L.,  1992).  Objectives  hierarchy  should  be  complete,  operational  (can  be  used 
in  analysis),  decomposable,  nonredundant  (no  double  counting)  and  minimal  (small 
dimensions)  (Kirkwood,  1997). 
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Attributes 


At  the  lowest  level  of  hierarchy,  all  of  the  objectives  must  have  an  attribute  in 
order  to  determine  level  of  objectives  achieved.  For  example,  number  of  fatalities  can  be 
an  attribute  for  “minimize  fatalities”  objective  (Bell,  Raiffa,  &  Tversky,  1988).  Attribute 
should  be  comprehensive,  operational  and  measurable.  Keeney  &  Raiffa,  1993  provides  a 
great  expalanation  of  these  features.  Some  of  the  attributes  have  direct  measurements. 
However,  if  we  have  subjective  attribute  scales,  like  prestige,  then  we  need  to  use  some 
proxy  attributes.  Proxy  attribute  measures  the  objective  indirectly  (Keeney  &  Raiffa, 
1993). 

Single-  Dimensional  Value  Functions 

Value  or  utility  functions  represent  decision  maker’s  preference  structure  that 
evaluates  alternatives.  These  functions  transform  the  attribute  levels  into  a  single 
dimensional  value  function  (SDVF)  by  considering  decision  maker’s  preferences.  By 
using  these  single  dimensional  measures,  alternatives’  overall  scores  become  independent 
from  units  (Bell,  Raiffa,  &  Tversky,  1988). 

Multiattribute  Value  Functions 

Since,  it  is  not  possible  to  satisfy  all  of  the  objectives  at  the  same  time  we  need  to 
define  value  tradeoffs.  Tradeoffs  are  subjective  and  differ  from  one  decision  maker  to 
another.  By  weighing  tradeoffs,  decision  maker  can  build  his  or  her  tradeoff  model.  By 
using  weights  and  single-attribute  value  functions,  we  can  calculate  the  overall  value 
score  for  each  alternative. 
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2.4.3  Steps  for  Value  Focused  Thinking 

VFT  necessitates  several  steps  to  construct  a  structure  on  decision  process.  Since, 
VFT  works  in  conjunction  with  MODA;  we  can  use  a  Multi  Objective  Decision  Analysis 
using  Value  Focused  Thinking  approach.  This  approach  necessitates  a  good 
communication  with  decision  makers,  stakeholders  and  subject  matter  experts.  Dillon- 
Merrill,  Parnell,  Buckshaw,  Casweel,  J.,  &  Hensley,  2008  summarizes  these  VFT  steps 
according  to  Kirkwood,  1997  as  shown  in  Figure  4. 


Figure  4  Steps  for  VFT  (Dillon-Merrill,  et  ah,  2008) 

Step  1:  Define  the  problem  and  identify  the  stakeholders. 

First,  we  analyze  qualitative  part  of  VFT,  which  includes  fundamental  objective, 
sub-objectives  and  value  measures.  We  start  our  analysis  by  problem  definition.  Problem 
definition  is  very  important  to  clearly  establishing  the  focus  of  the  decision  making 
process.  The  rest  of  the  process  depends  on  that  problem  definition  part.  After  defining 
problem,  we  need  to  identify  stakeholders  who  are  the  actors  that  can  influence  or  have 
effects  by  certain  problem  or  action.  For  big  projects,  we  can  face  many  stakeholders  and 
this  makes  decision-making  process  more  complex.  By  using  project  stakeholder 
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management,  we  can  make  this  problem  simpler.  Before  identifying  stakeholders,  an  easy 
way  to  analyze  them  is  to  separate  them.  Customer,  financial  institution  and  supplier  are 
some  of  the  internal  and  external  major  stakeholder  groups.  Project  stakeholder 
management  assumes  that  success  depends  on  taking  into  account  the  potential  impact 
decisions  on  all  stakeholders  during  the  entire  life  of  the  project  (Cleland  &  King,  1998). 
Step  2:  Identify  the  appropriate  objectives  for  the  problem  based  on  decision  makers  ’ 
and  other  stakeholders  ’  values  and  create  value  hierarchy. 

Generating  the  values  related  to  the  overall  objective  and  then  organizing  them 
into  a  value  hierarchy  is  the  next  step  after  clearly  identifying  the  problem.  In  order  to 
construct  a  successful  value  model  we  can  use  structured  techniques  for  value  modeling. 
There  are  four  structured  techniques  named  gold,  platinum,  silver  and  combined  standard. 
Gold  standard  model  depends  on  an  approved  vision,  policy,  strategy,  planning  or 
doctrine  document.  Platinum  standard  model  depends  on  interviews  with  decision  makers 
and  stakeholders.  Many  times  the  gold  standard  documents  are  not  adequate  and  we 
cannot  access  senior  decision  makers  and  stakeholders.  As  an  alternative,  the  silver 
standard  value  model  uses  data  from  the  stakeholder  representatives.  For  combined 
standard  model,  we  can  combine  standards  (Parnell,  2007). 

The  first  step  of  how  we  will  develop  the  value  model  is  identifying  the 
fundamental  objective.  Fundamental  objective  is  the  most  basic  objective  we  are  trying  to 
achieve.  Without  clear  and  precise  fundamental  objective,  it  is  difficult  to  make 
defensible  decisions.  By  specifying  area  of  concerns,  we  can  specify  objectives 
(Kirkwood,  1997).  After  that,  we  need  to  identify  the  sub-objectives  that  define  value. 
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Using  affinity  diagrams  is  a  good  way  to  determine  functions.  By  using  affinity  diagrams, 
we  can  obtain  mutually  exclusive  and  collectively  exhaustive  functions.  Once  set  of 
objectives  are  defined,  value  measures  are  required  for  evaluating  how  well  alternatives 
will  meet  each  objective  (Dillon-Merrill,  Parnell,  Buckshaw,  Casweel,  J.,  &  Hensley, 
2008). 

Step  3:  Develop  value  measures  and  value  functions  for  each  objective. 

After  completing  qualitative  analysis,  we  begin  quantitative  technique  that  uses 
value  functions,  weights  and  mathematical  equation  to  evaluate  the  alternatives.  We  first 
begin  with  value  functions. 

Each  value  measure  has  a  different  unit  of  measurement;  hence,  in  order  to 
compare  all  measures  conversion  to  the  same  scale  is  necessary.  Value  function  combines 
the  multiple  evaluation  measures  to  single  measure.  By  that  conversion,  we  can  scale  all 
measure  between  0  and  1  where  0  is  the  least  preferred  and  1  the  most  preferred.  We  can 
have  discrete  value  functions,  which  have  a  set  of  discrete  value  measures,  and 
continuous  value  functions,  which  have  a  continuous  range  of  value  measures. 
Continuous  value  functions  can  be  piecewise  linear  or  exponential.  A  piecewise  linear 
value  function  places  specific  relative  value  increments  for  all  possible  evaluation 
measures  (Kirkwood,  1997).  An  exponential  value  function  uses  a  mathematical  formula 
to  represent  a  continuous  range  of  value  increments.  This  strength  of  preference  functions 
depends  on  the  attribute  range  of  the  alternatives  in  the  decision  problem.  If  another 
alternative  enters  to  the  decision  problem  with  an  attribute  value  outside  of  the  current 
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range,  then  it  is  necessary  to  fonnulate  and  nonnalize  the  strength  of  preference  functions 
again. 

Step  4:  Identify  and  develop  alternatives  using  VFT. 

Next  step  is  alternative  generation.  One  of  the  most  important  parts  of  VFT  is 
alternative  generation  by  using  values.  Without  generating  appropriate  alternatives,  the 
decision  process  cannot  produce  a  good  result.  The  search  should  focus  on  the  values  in 
order  to  generate  better  alternatives  than  initial  alternatives. 

Step  5:  Asses  weights  for  each  measure  from  relevant  decision  makers  and  stakeholders. 

Then  we  can  assess  weights.  “Weights  are  required  to  tradeoff  the  objectives 
since  not  all  value  measures  will  be  equally  weighted  in  the  final  model  (Dillon-Merrill, 
Parnell,  Buckshaw,  Casweel,  J.,  &  Hensley,  2008)”.  Decision  maker  should  make  weight 
assessment.  Because  the  purpose  of  weighting  is  to  identify  which  value  measures  are 
most  important  to  decision  maker.  We  can  assign  weights  globally  or  locally.  Global 
weight  ensures  that  sum  of  each  measure  weight  which  is  at  the  lowest  tier  is  equal  to  1 . 
Local  weight  begins  from  the  top  of  the  value  hierarchy,  which  ensures  each  tier  sum  to 
be  equal  to  1.  By  multiplying  each  objective  in  the  same  branch,  we  may  find  associated 
global  weight. 

A  critically  important  task  in  decision-making  is  assigning  weights  to  criteria. 
Several  methods  have  been  proposed  to  weight  decision  criteria.  Indirect  weighting 
methods  identify  feasible  courses  of  action  open  to  decision  maker  to  define  a  set  of 
hypothetical  decisions.  One  of  them  is  Hypothetical  Equivalents  and  Inequivalents 
Method  (HEIM).  For  approximate  weighting  methods,  if  a  decision  maker  is  uncertain  as 
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to  her  tradeoffs,  attribute  weights  can  be  thought  of  as  sampled  from  a  distribution  of 
possible  weights.  Direct  criteria  weighting  methods  identify  weights  by  asking  the  direct 
value  of  that  weight.  Some  of  them  are  scaling,  swing  weighting  and  point  estimate 
method.  In  swing  weighing  method,  decision  maker  first  orders  the  criteria  in  terms  of 
their  importance  and  then  gives  weight  the  criteria.  As  an  assumption,  weights  reflect  the 
relative  importance  of  moving  an  attribute  from  worst  to  best  level  and  thus  are  defined 
on  a  ratio  scale.  Swing  weighting  method  has  more  potential  accuracy  than  other 
weighting  methods.  For  further  information,  refer  to  Jia,  Fischer,  &  Dyer,  1997  and  Nutt, 
1979. 

Step  6:  Score  and  rank  alternatives  on  each  of  the  value  measures  and  asses  the 
uncertainty  associated  with  the  scores. 

By  combining  value  functions  and  weights,  we  can  find  our  overall  scores  for 
each  alternative.  Additive  and  multiplicative  multiattribute  value  functions  are  two  types 
of  these  value  functions.  Additive  value  function  is  the  most  commonly  used 
multiattribute  value  function.  Since,  it  is  easy  to  implement.  In  addition,  it  reduces  the 
memory  storage  use.  It  assumes  that  single-attribute  value  score  has  weight  for  their 
importance  and  then  added  together.  Additive  value  function  gives  these  overall  scores 
by  the  following  formula. 

v(x)  =  ir=i»vi>£(x£)  =  1  (1) 

Where  V (x)  is  the  multi  objective  value  function,  p£(*£)  is  the  single  dimensional 
value  function  i,  w£  is  the  weight  of  the  evaluation  measure  (Keeney  &  Raiffa,  1993). 
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Step  7:  Analyze  the  sensitivity  of  the  analysis  to  assumptions  and  consider  refining 
alternatives  to  create  better  alternatives. 

After  scoring  the  alternatives,  a  sensitivity  analysis  should  determine  if  the 
preferred  recommendation  is  sensitive  to  small  changes  in  one  or  more  aspects  of  the 
model.  One  sensitivity  analysis  that  is  often  used  is  on  weights.  Because  the  model 
constructed  on  assumption  that  weighting  is  properly  given  and  reflect  decision  maker’s 
preferences.  If  a  model  is  detennined  to  be  highly  sensitive  to  a  particular  weight  or 
assumption,  the  decision  maker  may  wish  to  reconsider  carefully  that  aspect  of  the  model 
(Dillon-Merrill,  Parnell,  Buckshaw,  Casweel,  J.,  &  Hensley,  2008).  This  type  of 
sensitivity  analysis  begins  by  moving  a  selected  measure’s  weight  from  zero  to  one, 
regardless  of  the  predetermined  weight.  As  the  measure’s  weight  changes,  the  weights  of 
all  other  evaluation  measures  proportionally  adjust  to  ensure  all  weights  still  sum  to  one. 
Step  8:  Provide  recommendation  and  insights. 

Preparing,  presenting  and  documenting  the  recommendation  are  important  as 
problem  definition.  The  analyst  should  convey  useful  infonnation  to  decision  maker  with 
a  clear  and  concise  explanation.  We  cannot  expect  a  decision  maker  to  understand 
decision  analysis  language. 

2.5  Life  Cycle  Cost  Analysis  (LCCA) 

Many  projects,  especially  the  expensive  ones,  necessitate  a  good  cost  analysis.  If  only  the 
procurement  cost  is  considered,  it  is  not  likely  to  make  a  cost  effective  decision  in  the 
long  tenn.  Since,  most  of  the  studies  represent  that  costs  related  to  the  ownership  of  the 
system  exceed  costs  of  procurement  (Dhillon,  Life  Cycle  Costing  for  Engineers,  2010). 
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According  to  Ryan.W.J.,  1968  life  cycle  cost  can  be  10  to  100  times  of  procurement 
cost.  Without  a  proper  cost  analysis,  it  is  not  possible  to  visualize  all  of  the  costs  that  will 
affect  the  project  decision.  Therefore,  not  only  the  acquisition  cost  but  also  other  costs,  as 
illustrated  in  Figure  5,  need  additional  interest  while  making  decisions. 


Poor  Management 


Figure  5  The  Problem  of  the  Total  Cost  Visibility  (Fabrycky,  J.,  &  Blanchard,  1991) 
Fuller  &  Petersen,  1996  defines  LCCA,  as  “LCCA  is  an  economic  method  of 
project  evaluation  in  which  all  costs  arising  from  owning,  operating,  maintaining  and 
ultimately  disposing  of  a  project  are  considered  to  be  potentially  important  to  that 
decision”.  Alternatively,  simply  “Life  cycle  cost  of  a  system  can  be  defined  as  the  sum  of 
all  costs  incurred  during  its  life  span”  (Dhillon,  Life  Cycle  Costing  for  Engineers,  2010). 
The  main  objective  of  using  LCCA  is  minimizing  the  total  cost  of  a  project  over  its 
lifetime  (Remer,  1977). 

LCCA  has  many  advantages.  It  is  useful  to  reduce  total  cost  of  the  system  and  to 
control  overall  process  of  the  program.  It  is  an  excellent  tool  for  selecting  among  the 
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competing  alternatives  and  contractors.  In  addition  to  these  advantages,  LCCA  may 
reduce  the  costs  related  to  ownership.  By  analyzing  the  future  costs,  we  determine  the 
problematic  sides  of  the  project  that  increase  the  future  costs.  Hence,  we  can  define  new 
solutions  to  solve  those  problems  (Remer,  1977).  In  addition  to  these  advantages,  LCCA 
has  some  disadvantages.  Estimating  the  project  lifetime  is  very  time  consuming  and 
expensive;  and  inaccurate  lifetime  estimation  may  be  misleading.  Accuracy  of  the  data 
directly  effects  the  decision;  hence,  data  may  be  another  burden  to  worry  about  the 
accuracy  of  the  analysis  (Dhillon,  2010). 

Many  companies  and  governmental  organizations  use  life  cycle  cost  techniques  to 
evaluate  alternatives  and  to  make  cost  effective  decisions.  There  are  many  applications 
using  life  cycle  cost  analysis.  Figure  6  (Sage  &  Rouse,  1999)represents  some  of  the 
application  areas  of  life  cycle  cost. 


Figure  6  Life  cycle  cost  applications  (Sage  &  Rouse,  1999) 
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2.5.1  Life  Cycle  Cost  Elements 

Life  cycle  cost  has  four  elements  based  on  the  organizational  activity  over  the  life 
cycle.  These  elements  are  research  and  development  (R&D),  investment,  operating  and 
support  (O&S),  and  disposal.  These  elements  are  essential  for  cost  estimation  of  the 
projects.  Figure  7  is  an  illustration  of  these  elements  and  their  effects  on  the  project’s  life 
cycle  cost.  This  illustration  may  vary  for  different  projects  (Fabrycky,  J.,  &  Blanchard, 
1991). 


ENGINEEIUNG/ 
MANUFACTURING 
DEVELOPMENT  PHASE 


Figure  7  Project  Life  Cycle  (illustrative)  (Cost  Analysis  Improvement  Group,  1992) 
Research  and  Development  Cost  (R&D  Cost) 

R&D  cost  consists  of  costs  stem  from  feasibility  studies;  market  analysis; 
modeling;  trade-off  analyses;  engineering  design;  development;  software;  system  tests 
and  evaluation;  design  data  and  documentation. 
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Investment  Cost 


Investment  cost  is  associated  with  producing,  procuring,  and  deploying  system, 
initial  logistic  support  requirements,  training,  data,  initial  spares,  and  facility 
construction. 

Operation  and  Support  Cost  (O&S  Cost) 

O&S  cost  incurred  from  the  costs  related  to  operating,  maintaining  and  supporting 
the  system.  It  includes  costs  for  personnel,  consumable  and  repairable  material, 
organizational,  intermediate  and  depot  maintenance,  and  all  appropriate  levels  of 
maintenance,  facilities,  and  sustaining  investment. 

Disposal  Cost 

Disposal  cost  includes  costs  associated  with  deactivating  a  system  at  the  end  of  its 
useful  life.  Because  of  not  having  a  significant  effect  on  life  cycle  cost,  it  is  generally 
excluded  from  most  of  the  studies  except  nuclear  waste  and  similar  disposal  problems. 

2.5.2  Steps  for  LCCA 

LCCA  provides  a  systematic  approach  to  obtain  a  cost  effective  solution.  The 
analysis  process  is  iterative  and  applicable  to  any  phase  of  the  system  life  cycle.  Figure  8 
illustrates  this  process. 

1.  Planning  2.  Definition  3.Develpoment  N.  4.  Analysis  5.  Reporting 

LCC  objective,  System,  use,  N.  CBS,  estimation  Sensitivity  analysis,  risk  Interpretation  of 

y  alternatives,  data  /  maintenance  /  method,  data  /  analysis,  uncertainty  /results,  capturing  of 

/  availability  /  /  collection  /  analysis  /  results 

Figure  8  Steps  in  a  Life  Cycle  Cost  Analysis  (Bassford,  Voort,  Boer,  &  Pung,  2008) 
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Step  1:  Planning 

The  first  and  most  essential  step  is  planning.  Since,  without  clear  identification  of 
the  problem  and  LCCA  objective,  it  is  possible  to  make  inefficient  decisions.  In  addition, 
the  rest  of  the  process  depends  on  that  problem  and  objective  definition  step.  Since,  clear 
problem  definition  reveals  clear  cost  breakdown  structure.  On  the  other  hand, 
identification  of  the  alternatives  is  important.  The  analyst,  decision  maker  and 
stakeholders  should  define  operational  requirements  and  maintenance  concepts  of 
alternatives  to  be  a  candidate  of  winning  alternative.  Since  decisions  are  made  by 
analyzing  desired  characteristics  and  performance  and  the  cost  of  the  system.  After  that, 
lifecycle  cost  phases  and  their  related  costs  need  to  be  examined  in  general  and  searched 
for  available  data.  If  data  is  not  available  and  that  is  not  determined  at  the  beginning  of 
the  analysis,  it  is  impossible  to  accomplish  analysis. 

Step  2:  Definition 

The  purpose  of  LCCA  is  detennining  the  most  cost  efficient  alternative  of  a 
specified  system.  Therefore,  system  identification  is  very  important  in  order  to  construct 
a  systematic  relation  with  outside  effects  and  internal  subsystems.  With  the  intention  of 
specifying  subsystems,  the  definition  of  system  and  its  purpose  of  use  are  necessary. 
After  these  definitions  maintenance  necessitations  becomes  clear  for  further  break  down. 
Step  3:  Development 

The  next  step  is  to  develop  a  Cost  Breakdown  Structure  (CBS)  for  evaluating  the 
life  cycle  cost  of  alternatives.  CBS  constructs  a  systematic  relation  and  communication 
between  objectives,  costs  and  analysis.  Main  goal  for  creating  CBS  is  to  visualize  hidden 
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costs  (Fabrycky,  J.,  &  Blanchard,  1991).  Systems  are  different  from  each  other;  hence, 
every  system  has  a  different  CBS.  However,  Figure  9  can  be  a  general  start  point  to 
develop  a  CBS.  Gaps  related  to  each  LCCA  element  are  defined  by  system  and 
associated  costs  of  it.  For  CBS  of  different  systems  refer  to  Fabrycky,  J.,  &  Blanchard, 
1991. 


Figure  9  A  Start  Point  for  Cost  Breakdown  Structure  (Fabrycky,  J.,  &  Blanchard,  1991) 
Every  system  has  its  own  features  including  different  subsystem,  inputs  and 
outputs,  users,  equipments  etc.  Therefore,  it  is  not  possible  to  define  only  one  general 
cost  model  for  different  systems.  Cost  model  is  critical  step  for  life  cycle  cost  analysis; 
since,  if  the  model  is  not  representing  the  real  system,  the  analyst  cannot  obtain  a  correct 
result  from  the  analysis.  Literature  has  diverse  numbers  of  cost  models;  so,  selecting  a 
suitable  model  for  the  system  necessitates  attention.  Dhillon,  1989  classifies  these  models 
with  two  categories  that  are  general  life  cycle  cost  models  and  specific  life  cycle  cost 
models.  Dhillon,  1989  includes  many  examples  about  life  cycle  cost  models  of  different 
equipments,  particular  and  general  systems. 

By  using  the  CBS,  analyst  can  proceed  with  the  cost  estimation.  Cost  estimation 
should  be  very  close  to  real  data  in  order  to  make  a  suitable  and  reliable  analysis. 
Because  of  missing  data,  generally  the  experience  is  necessary  to  calculate  the  cost 
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estimates.  Therefore,  it  becomes  a  challenging  step  in  the  life  cycle  process.  There  are 
many  cost  estimation  methods  available  in  the  literature.  According  to  Sage  &  Rouse, 
1999,  these  estimations  can  be  made  by  analogy,  expert  judgment,  bottom-up  and 
parametric  models. 

Estimation  by  analogy  uses  data  from  past  system.  If  the  data  is  robust,  this 
method  is  easy  to  imply.  Otherwise,  the  estimation  that  affects  analysis  directly  will  be 
unreliable.  Expert  judgment  method  uses  expert  view  due  to  their  experience  to  estimate 
cost.  Nevertheless,  unreliable  experts  and  new  system  that  cannot  yield  experience  are  the 
problems  of  this  method.  Bottom-up  model  estimates  the  cost  of  subsystems  and 
accumulates  to  find  the  overall  cost  of  the  system.  It  necessitates  data  collection  from 
subsystem  responsible.  Therefore,  this  is  a  very  time-consuming  and  people-dependent 
method  (Sage  &  Rouse,  1999).  Parametric  model  uses  statistical  techniques.  It  is  possible 
to  calculate  total  cost  by  the  relation  function  of  different  components  and  their  statistical 
data.  Yet,  it  is  generally  necessary  to  use  this  method  in  conjunction  with  analogy  and 
expert  judgment  methods  (Fabrycky,  J.,  &  Blanchard,  1991). 

Project  life  lasts  for  a  period.  Hence,  time  is  an  input  for  analysis.  In  order  to 
consider  time  in  the  analysis,  it  is  essential  to  develop  cost  profile  of  the  system.  Cost 
profile  is  the  graphical  or  tabular  representation  showing  the  distribution  of  costs  over  the 
life  cycle  of  a  product  (IEC60300-3-3:  Life  cycle  costing).  By  using  the  CBS  and 
estimated  costs,  it  is  possible  to  present  the  cost  of  each  activity  year  by  year.  Monetary 
value  represents  the  purchasing  power  of  the  money  for  the  current  day. 
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Step  4:  Analysis 

In  order  to  compare  alternatives,  analyst  should  make  an  economic  evaluation  by 
using  the  cost  profile.  The  main  concern  is  to  find  a  desirable  alternative  among  the 
possible  ones.  We  can  use  a  discounted  profile  by  time  value  of  money  in  order  to 
compare  different  alternatives.  Since  today’s  value  of  money  is  not  equal  to  the  value  of 
future  money.  The  main  idea  is  discounting  the  future  value  to  the  present  value. 
Assumption  of  the  method  is  the  known  discount  rate.  Many  companies  and  governments 
specify  their  discount  rates  to  use  in  their  economic  evaluations.  Present  equivalent 
evaluation,  annual  equivalent  evaluation,  future  equivalent  evaluation,  rate  of  return 
evaluation  and  pay  out  evaluation  are  some  of  the  methods  that  use  the  time  value  of 
money. 

Another  way  to  evaluate  alternatives  is  using  budget  profile  with  constant  money 
value  by  year-by-year  basis.  Monetary  value  will  be  in  terms  of  today  value. 
Nevertheless,  it  is  suitable  for  only  one  cost  profile  evaluation.  By  applying  the  effects  of 
learning  curves  and  inflation  factor  to  the  budgetary  profile,  we  can  obtain  a  different 
way  to  evaluate  single  cost  profile  (Fabrycky,  J.,  &  Blanchard,  1991). 

Following  these  calculations,  a  break-even  analysis  may  be  suitable  in  order  to 
detennine  the  preferred  alternative.  Cumulative  life  cycle  cost  for  program  time 
visualizes  the  alternatives  perfonnances  in  terms  of  their  life  cycle  cost.  If  it  is  planned  to 
use  a  system  more  than  the  break-even  point  time,  it  will  be  suitable  to  choose  the  one 
with  the  less  cost  after  that  break-even  point. 
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In  order  to  reduce  the  life  cycle  cost  and  improve  the  system,  the  analyst  should 
define  high-cost  contributors.  High-  cost  contributors  are  the  elements  of  LCCA  that  has 
a  great  impact  on  system.  By  using  CBS  and  cost  estimates  of  each  element,  it  is  possible 
to  identify  the  contributors  that  increase  cost.  Identification  of  high-cost  contributors 
reveals  specification  of  their  causes.  Once  these  causes  are  defined,  it  is  possible  to 
identify  other  effected  system  contributors. 

Decisions  are  subject  to  uncertainty.  LCCA  decreases  the  level  of  that 
uncertainty;  yet,  it  still  includes  some  issues  of  uncertainty.  It  is  crucial  to  check  LCCA 
results  if  they  are  sensitive  to  change  of  major  inputs.  Sensitivity  analysis  is  a 
detenninistic  technique  that  estimates  how  changes  in  the  input  will  change  the  outcome. 
It  identifies  the  inputs  that  have  the  greatest  effect  on  the  system.  It  is  possible  to 
calculate  the  upper  and  lower  bounds  of  LCC  by  the  range  of  outcomes.  By  simply 
changing  the  value  of  each  input  and  keeping  all  others  constant,  change  in  LCC  can  be 
measured  (Fuller  &  Petersen,  1996). 

Risk  analysis  provides  an  overview  of  what  are  the  consequences  if  a  decision 
does  not  proceed  according  to  program.  In  order  to  make  suitable  recommendations, 
analyst  should  consider  the  risk  as  a  part  of  analysis.  Risk  and  costs  are  dependent 
concepts;  because,  if  risk  increases,  cost  increases  as  well.  In  addition,  if  cost  is  not 
controlled,  risk  of  unsuccessful  project  arises.  Therefore,  it  is  essential  to  identify  and 
eliminate  the  potential  risk  areas  early.  High  cost  contributors  and  their  causes  are  the 
main  potential  areas  of  risk  in  LCC  (Sage  &  Rouse,  1999). 
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Step  5:  Reporting 

Reporting  LCCA  results  to  decision  maker  is  essential.  Since,  the  decision  is 
dependent  to  analyst  presentation.  An  executive  summary  should  be  included  to  provide 
an  overview  of  analysis.  The  overview  should  explain  basic  concepts  including 
definitions,  objectives,  assumptions  and  constraints.  He/she  will  not  be  familiar  with  most 
of  the  concepts  that  the  analysis  utilizes.  CBS,  explanation  of  elements  and  cost 
estimation  method  should  be  clear.  High-cost  drivers,  sensitivity  and  other  analysis 
should  be  in  the  report.  Regarding  the  analysis,  results  conclusions  and  recommendations 
should  be  included  for  LCCA  report  (IEC60300-3-3:  Life  cycle  costing). 


35 


III.  Methodology 


This  chapter  discusses  the  first  five  steps  of  the  eight-step  VFT  process  and  the  first 
three  steps  of  the  five-step  LCCA  process.  The  last  three  steps  are  included  in  the 
analysis  and  recommendation  sections  of  this  research. 

3.1  VFT  Steps 

3.1.1  Stepl:  Problem  definition  and  stakeholders’  identification: 

Aerobatic  display  team  is  the  one  of  the  most  important  elements  of  any  Air  Force 
all  around  the  world.  Since,  it  is  the  international  presenter  of  Air  Force  and  country.  So 
selecting  aerobatic  aircraft  has  big  importance.  Therefore,  it  necessitates  a  good 
identification  of  stakeholders. 

Stakeholders  of  this  decision-making  process  split  into  three  groups:  Pilots  who 
are  the  actual  users  of  the  aircraft  during  flight,  maintenance  personnel  who  are  the  actual 
user  of  the  aircraft  after  landing  and  audiences  who  are  the  main  purpose  of  the  display. 
The  main  purpose  of  a  system  is  to  satisfy  stakeholders’  demands  with  appropriate  design 
while  realizing  objectives  optimally.  For  this  research,  constructer  is  not  detennined  as  a 
stakeholder.  It  is  assumed  that  the  available  producers  could  satisfy  any  demand  by 
stakeholders. 

As  the  decision  maker,  the  requirement  officer  is  chosen.  A  pilot  who  is  a  four- 
wing  leader,  a  maintenance  officer  who  is  the  maintenance  team  chief  and  audience  who 
are  the  general  Turkish  people  are  designated  as  stakeholders. 
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3.1.2  Step  2:  Identification  of  objectives  and  functions 

Among  the  four  different  structured  techniques  for  value  modeling,  silver 
standard  model  is  selected  to  develop  hierarchy.  So  interviews  made  with  decision  maker 
and  stakeholders. 

The  fundamental  objective  is  “Selecting  the  most  appropriate  aircraft  which  can 
supply  best  representation  of  Turkish  Air  Force  and  Republic  of  Turkey  all  around  the 
world”.  This  is  a  clear  and  concise  statement  of  the  most  basic  reason  for  decision. 

After  making  interview  with  pilot,  maintenance  officer  and  audience,  functions 
are  designated  by  grouping  all  of  the  ideas  in  order  to  obtain  mutually  exclusive  and 
collectively  exhaustive  sub-objectives.  Three  sub  objectives  emerged  from  grouping 
process:  maximize  flight  performance,  minimize  withdrawals  of  logistics  issues  and 
maximize  prestige.  Figure  10  illustrates  fundamental  objective  and  sub-objectives. 


Figure  50  Aerobatic  Aircraft  Selection  Value  Hierarchy. 
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3.1.3  Step  3:  Develop  value  measures  and  value  functions  for  each  objective. 

Once  the  set  of  objectives  is  defined,  value  measures  are  required  for  evaluating 
how  well  alternatives  will  meet  each  objective.  Value  measures  are  defined  for  each 
objective.  Overall  value  hierarchy  is  in  Figure  1 1. 


Selecting  the 
Best  Aerobatic 
Aircraft 


Figure  61  Overall  Value  Hierarchy 

Endurance:  It  is  the  length  of  time,  which  an  aircraft  can  fly  without  refueling.  An 
aerobatic  aircraft  may  perform  shows  away  from  runways.  Moreover,  a  spare  aircraft 
always  waits  in  case  of  a  problem  of  other  aircrafts.  In  addition,  during  international 
deployment  for  the  shows,  it  is  beneficial  to  make  less  air  refueling  activity  concerning 
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safety  and  cost.  Therefore,  an  aerobatic  aircraft  should  have  a  high  endurance.  Minutes 
express  the  measurement  of  endurance. 

Maneuverability  is  not  a  part  of  the  hierarchy.  Since,  all  of  the  alternatives  are 
selected  by  considering  requirements  of  an  aerobatic  show. 

Thrust/Weight  Ratio:  Thrust  is  the  propulsion  of  engine.  However,  it  is  not  a  significant 
measure  to  evaluate  aircraft  perfonnance.  Therefore,  thrust  to  weight  ratio  is  a  better 
performance  measurement.  It  is  directly  proportional  to  the  acceleration  of  the  aircraft. 
An  aircraft  with  a  high  thrust  to  weight  ratio  has  high  acceleration.  An  aerobatic  display 
aircraft  should  have  a  high  thrust  to  weight  ratio  in  order  to  perform  high  performance 
maneuvers. 

Air  Refueling  Capability:  Air  refueling  is  the  process  of  transferring  fuel  from  the  tanker 
aircraft  to  the  receiver  aircraft  during  flight  (Bolkcom,  2006).  International  deployments 
necessitate  air-refueling  capability.  Because,  without  air  refueling  many  landings  and 
take-offs  are  necessary.  Every  landing  and  take-off  decrease  the  flight  safety  and  increase 
cost.  Therefore,  an  aerobatic  display  aircraft  should  have  air-refueling  capability. 

Ease  to  Repair:  Mean  time  to  repair  (MTTR)  which  is  a  basic  measure  of  the 
maintainability  and  reliability  of  a  system  and  is  a  direct  measure  for  ease  to  repair.  It 
represents  the  average  time  required  to  complete  a  corrective  maintenance  action  for  each 
failure.  Corrective  maintenance  actions  occur  because  of  an  actual  or  suspected  failure  to 
restore  an  item  to  a  specified  condition.  Hours  express  the  measurement  of  ease  to  repair. 
Equation  2  illustrates  how  to  calculate  MTTR. 

|#rrrjT  n  _  Total  Corrective  Maintenance  Time 

1*1 1  1  tv  —  :  I  ~  ”  ( Z ) 

Total  Number  of  Failures 
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An  aircraft  should  have  a  low  MTTR,  which  means  less  work,  time  consumption  and 
more  availability. 

Ease  to  Prepare:  For  each  sortie,  preflight  and  post  flight  maintenance  is  necessary  for 
every  aircraft.  On  the  other  hand,  in  order  to  maintain  an  aircraft  some  preventive 
maintenance  procedures  should  apply.  Preventive  maintenance  is  the  systematic  care, 
servicing  and  inspection  of  equipment  to  improve  reliability  and  increase  system  lifetime. 
Lombardo,  1998  includes  examples  of  preventive  maintenance.  It  can  directly  be 
measured  by  maintenance  man-hour  per  flight  hour  scheduled  (MMH/FH(S)).  Equation  3 
shows  how  to  calculate  it  MMH/FH(S). 


MMH/FH(S)  = 


Total  Preventive  Man-Hours 
Total  Flight  Hours 


(3) 


An  aircraft  should  have  a  low  MMH/FF1(S)  that  means  less  work,  less  time 
consumption  and  more  availability. 

Maintenance  Personnel  Requirement:  It  is  total  number  of  maintenance  personnel 
needed  to  support  20  aircraft.  Fewer  personnel  are  better. 

Skill  Level:  Maintenance  personnel  level  of  skills  is  crucial  to  support  aircraft  and  play 
an  important  role  in  the  Air  Force’s  ability  to  accomplish  its  mission.  Turkish  Air  Force 
uses  three  different  skill  levels  that  are  level-3,  level-5  and  level-7.  Level-3  may 
accomplish  the  tasks  unsupervised.  Level-5  may  be  assigned  to  quality  assurance 
positions.  Level-7  needs  to  satisfy  different  supervisory  and  management  positions  such 
as  shift  leader  and  section  chief.  TurAF  assigns  different  skill  levels  percentage  for 
different  aircrafts  depending  on  the  complexity  of  the  aircraft.  If  an  aircraft  is  complex,  it 
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necessitates  more  level-5  and  level-7  personnel  which  means  increasing  cost,  increasing 
time  consumption  and  extra  effort  to  educate  personnel. 

Yet,  every  country  has  its  own  maintenance  skill  level  system.  Hence,  we  cannot 
directly  use  TurAF  skill  level  categories.  It  is  possible  to  measure  that  by  using  skill 
levels  low,  medium  and  high.  By  examining  the  skill  level  systems  of  candidate 
countries,  it  is  possible  to  group  maintenance  personnel.  Percentage  of  maintenance  of 
each  skill  level  to  support  20  aircraft  is  the  measure  for  skill  level.  This  measure  is  proxy 
to  determine  the  complexity  of  the  aircraft.  Less  high  skill  level  personnel  is  better. 
Aircraft  Prevalence:  If  an  aircraft  is  commonly  used,  then  material  supply  will  be  easier 
and  cheaper  because  of  the  mass  production  of  parts.  Therefore,  total  number  of  aircraft 
used  all  around  the  world  is  a  measure  for  aircraft  prevalence. 

Leasing  Right  to  Produce:  If  agreements  maintain  leasing  right  to  produce  for  a  specific 
amount,  then  it  will  be  easier  to  obtain  supply  materials.  The  right  of  production 
percentage  of  each  aircraft  can  measure  it. 

Type  of  Procurement:  As  explained  in  Turkish  Procurement  Law,  there  are  different 
types  of  procurement  agreements.  If  agreement  is  domestic,  it  will  be  faster  and  cheaper 
to  obtain  material  support.  If  agreement  is  international  FMS  and  NAMSA  procurement 
procedures  will  be  fast,  but  expensive.  On  the  other  hand,  if  international  agreement  is 
MO  A,  it  will  be  cheaper  to  obtain  materials,  but  it  will  take  long  time  to  get  them.  If  PBL 
(Performance  Based  Logistics)  is  type  of  the  international  agreement,  it  will  also  be 
cheap  and  fast,  but  not  as  fast  and  cheap  as  domestic  agreement.  Therefore,  we  can 
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categorize  them  domestic  procurement,  procurement  by  PBL,  procurement  by  FMS, 
procurement  by  NAMSA  and  procurement  by  MOA. 

Life  of  Project:  It  is  the  planned  time  for  the  production  of  the  aircraft.  It  should  be 
greater  than  30  years,  which  is  the  period  of  usage  of  these  aircrafts  by  TurAF.  By 
looking  at  the  aircraft  production  scheduling,  we  can  decide  if  the  production  stops  in 
next  30  years.  This  measure  detennines  the  continuity  of  supply. 

Ease  to  Transport:  Because  of  not  being  on  the  home  base,  each  show  necessitates 
specific  amount  of  spare  parts  and  support  equipment.  Turkish  Stars  aerobatic  display 
team  has  one  C-130  for  domestic  and  one  C-160  for  international  and  domestic  material 
transportation.  C-130  maximum  number  of  pallet  is  eight  and  C-160  maximum  number 
of  pallet  is  six.  Number  of  pallets  change  due  to  the  location  of  the  show.  International 
shows  necessitate  more  pallets  then  domestic  shows.  Since  Turkish  Stars  deploys  all 
around  the  world  many  times,  it  is  desirable  to  carry  few  pallets.  Therefore,  an  aerobatic 
aircraft  should  necessitate  fewer  pallets  to  transport  to  decrease  effort  for  deployment 
preparation  and  cost.  Number  of  pallets  for  international  and  number  of  pallets  for 
domestic  shows  are  the  measures  of  ease  to  transport. 

Origin  of  Aircraft:  The  purpose  of  Turkish  Star  is  to  present  the  TurAF,  Turkish 
Republic,  to  contribute  to  the  efficiency  and  discipline  of  the  TurAF,  to  reinforce  the 
people  trust  in  the  Air  Force,  and  to  increase  the  interest  of  the  air  force  among  the  young 
people  of  Turkey.  Hence,  people  of  Turkey  should  approve  the  Turkish  Star’s  aircraft.  As 
a  result,  the  origin  of  the  country  becomes  an  effect  for  selecting  the  aircraft.  Categories 
for  the  countries  represent  the  measure  for  origin  of  the  aircraft.  Cat-1  represents  Turkey, 
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cat-2  represents  ally  countries,  cat-3  represents  countries  with  good  relationship  and  cat-4 
represents  other  countries. 

Aircraft  Type:  For  aerobatic  teams,  type  of  the  aircraft  is  an  important  issue.  Since,  it 
represents  the  capability  of  the  aerobatic  display  team  and  air  force.  On  the  other  hand, 
type  of  the  aircraft  is  crucial  for  attracting  young  people.  Jet  aircraft  and  propeller  aircraft 
categories  are  measures  for  aircraft  type. 

After  constructing  the  hierarchy,  the  next  step  is  to  create  the  value  functions  for 
each  evaluation  measure.  As  stated  in  previous  chapters,  it  is  necessary  to  convert  all 
evaluation  measures  to  unit  less  measures  between  zero  and  one  where  one  is  best  and 
zero  is  the  worst  score.  Value  functions  are  constructed  with  the  inputs  of  SMEs  and 
decision  maker.  A  computer  software  program  called  Hierarchy  Builder  (Weir,  2008) 
was  used  to  simplify  the  process  to  create  value  functions.  Hierarchy  Builder  is  verified 
by  previous  studies.  For  detailed  infonnation,  refer  to  (Malyemez,  2011).  First,  the 
decision  maker  and  SMEs  decided  whether  a  measure  has  discrete  or  continuous  scale.  If 
a  measure  was  defined  as  discrete,  all  categories  of  the  measure  were  defined  and  given  a 
value.  If  a  measure  was  continuous,  decision  maker  and  SMEs  were  asked  to  specify  if  it 
has  an  increasing  or  decreasing  value  function.  For  an  increasing  value  function,  the 
minimum  score  has  value  of  zero  and  maximum  score  has  value  of  one.  On  the  other 
hand,  for  a  decreasing  value  function,  the  minimum  score  has  value  of  one  and  the 
maximum  score  has  a  value  of  zero.  After  that,  the  upper  and  lower  bounds  were 
specified  the  least  and  the  most  preferred  score  of  the  measure.  The  reference  point  0.8 
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was  used  to  determine  the  shape  of  the  value  functions  which  means  at  what  score  you 
obtain  the  0.8  value  compared  to  one.  All  functions  are  shown  in  Appendix-A. 

3.1.4  Step  4:  Identify  and  develop  alternatives  using  VFT. 

After  considering  a  decision  maker’s  values  and  a  value  hierarchy,  it  is  suitable  to 
generate  alternatives.  New  alternatives  should  be  better  than  the  current  aircraft.  Possible 
candidates  to  be  aerobatic  demonstration  aircraft  of  Turkish  Stars  and  their  performance 
measures  are  generated  by  using  data  of  different  aircrafts.  The  aircrafts  that  are  data 
source  of  generated  aircrafts  are  T-50  Golden  Eagle,  F-16  Fighting  Falcon,  Euro-Fighter 
Typhoon,  Mirage-2000,  Sukhoi  30  MKI,  Gripen  JAS39B,  F-35  Joint  Strike  Fighter, 
Alpha  Jet,  HAWK,  KT-1T,  Cessna  208  Caravan  and  T-6.  Data  for  each  alternative  with 
related  value  measures  are  shown  in  Table  B.  1. 

3.1.5  Step  5:  Asses  weights  for  each  measure  from  relevant  decision  makers  and 
stakeholders. 

Weights  are  assigned  to  find  the  more  important  measure;  since,  the  larger  the 
weight,  the  bigger  the  effect  of  that  measure.  In  order  to  asses  weights  swing  weighting 
method  is  used.  Decision  makers  were  asked  to  rank  three  sub-objectives  performance, 
logistics  and  prestige  and  give  points  from  zero  to  100  according  to  their  importance  to 
least  preferred  one.  After  that  step,  the  points  are  nonnalized  to  sum  to  one.  For  each  sub¬ 
objective  and  measures,  these  procedures  are  repeated.  Table  1  represents  the  results  of 
the  process. 
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Table  1  Local  and  Global  Weights  of  the  Value  Hierarchy 


3.2  LCCA  Steps 
3.2.1  Step  1:  Planning 

The  objective  of  LCCA  for  this  research  is  to  calculate  appropriate  cost  values  for 
candidate  alternatives.  By  this  planned  approach,  it  is  not  likely  to  miss  important  cost 
factors.  TurAF  seeks  for  the  most  efficient  aircraft  that  means  the  aircraft  that  creates  the 
highest  score  with  the  appropriate  cost.  Objectives  and  fourteen  different  alternatives 
considering  these  objectives  are  determined  by  using  VFT  by  including  decision  maker, 
stakeholders  and  their  requirements. 

Data  gathering  is  the  most  challenging  part  of  LCCA.  Hence,  the  availability  of 
the  data  must  be  verified.  In  this  research  real  world  data  of  different  aircrafts  was  used  to 
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generate  cost  data  for  all  aircrafts.  Therefore,  data  may  not  be  accurate.  Because  the 
purpose  of  this  research  is  to  construct  a  decision  model;  not  making  a  decision 
recommendation.  TurAF  procurement  procedure  requires  availability  of  the  data.  Hence, 
before  analysis,  producers  of  the  candidate  aircrafts  must  supply  the  desired  information 
and  data.  During  the  real  procurement  procedure,  all  data  will  be  available. 

3.2.2  Step  2:  Definition 

TurAF  plans  to  procure  20  aerobatic  aircrafts  for  30  years  period.  TurAF  will  not 
produce  the  aircrafts  and  their  related  systems;  thus,  outside  sources  including  domestic 
and  foreign  producers  will  provide  the  aircrafts  and  their  related  systems.  Procured 
aircrafts  will  be  used  for  aerobatic  display  purpose  in  domestic  and  international  shows. 
TurAF  will  provide  flight  duty  and  maintenance  service  of  all  of  the  aspects  of  the 
system.  Therefore,  flight  and  maintenance  personnel  need  training. 

3.2.3  Step  3:  Development 

The  next  step  is  to  develop  a  CBS  for  evaluating  the  life  cycle  cost  of  alternatives 
to  visualize  hidden  costs.  CBS  must  include  consideration  of  all  costs.  A  typical  fighter 
aircraft  CBS  has  three  categories  which  are  R&D  cost,  procurement  cost  and  O&S  cost. 
As  stated  before,  every  system  has  its  own  features  and  it  is  not  possible  to  define  only 
one  general  cost  model  for  different  systems.  TurAF  will  not  produce  the  aircrafts  and 
their  related  systems.  Hence,  this  research  will  consider  only  the  procurement  cost  and 
operations  and  support  cost.  Consequently  the  model  is: 
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LCCfa:Life  cycle  cost  of  a  typical  fighter  aircraft. 

PC fa\  Fighter  aircraft  procurement  cost. 

OSCfa: Fighter  aircraft  operations  and  support  cost. 

The  CBS  assumed  for  this  research  is  illustrated  in  Figure  12.  Not  all  of  the  cost 
components  are  critical.  The  next  step  will  be  considering  high  cost  contributors  of  the 
LCC. 
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Figure  72  Cost  breakdown  structure 


Procurement  Cost  Contributors 


Aircraft  cost  includes  unit  acquisition  cost  of  20  aircrafts. 

Aircraft  related  equipment  cost  includes  equipments  that  are  necessary  to  operate 
aircrafts.  So,  these  equipments  should  be  purchased  with  aircraft. 

Updates  during  production  cost  include  requirements  of  TurAF  and  system  enhancement 
costs  caused  by  project  design. 

Modification  for  airs  how  cost  includes  special  modifications  of  the  aircraft  to  use  it  as  an 
aerobatic  aircraft.  For  instance,  smoke  production  system  is  a  part  of  aerobatic 
demonstration  aircraft. 

Initial  spares  and  inventory  cost  includes  the  procurement  of  units  to  support 
maintenance  activities. 

Special  tools  and  test  equipments  cost  includes  testing  equipment  for  repaired  parts  and 
parts  that  are  in  the  inventory.  These  equipments  determine  the  functionality  of  parts. 
Documentation  and  technical  data  includes  the  preparation  and  publication  of  system 
installation  and  test  instructions,  operating  and  maintenance  procedures.  It  is  crucial  to 
operate  and  maintain  the  system  during  its  lifetime. 

Initial  training  includes  training  of  operating  and  maintenance  personnel  to  operate  and 
maintain  aircrafts.  When  a  system  is  first  introduced,  training  of  operating  and 
maintenance  personnel  is  necessary. 

Maintenance  facilities  cost  includes  modification  of  the  current  facilities  and  construction 
of  new  facilities. 
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Simulators  includes  simulator  and  its  related  facility  construction  and  modification  of  the 
current  facilities. 

O&S  Cost  Contributors 

Operating  personnel  cost  includes  the  total  costs  of  operating  personnel  depending  on 
type  of  duty. 

Operator  training  includes  training  of  new  operators.  In  addition,  system  changes 
necessitate  additional  training  during  the  lifetime  of  a  system. 

Operational  facilities  cost  includes  operating  cost  for  simulators  and  operating  personnel 
training  facilities. 

Infrastructure  for  use  of  materials  includes  operating  cost  of  maintenance  facilities. 

Fuel  cost  includes  cost  of  necessary  fuel  to  fly  aircrafts  for  training,  performing  and 
deployment. 

Sustainment  cost  includes  all  periodical  adjustments  and  inspections,  preventive 
maintenance,  minor  and  major  repairs  and  replacements  of  components. 

Spares  and  inventory  support  cost  includes  cost  of  spare  parts  and  their  inventory.  Spare 
parts  are  essential  to  increase  the  availability  of  the  system.  In  addition,  the  inventory  is 
important;  since,  if  inventory  level  is  higher  than  necessary,  then  it  will  increase  the  cost. 
On  the  other  hand,  if  the  level  is  not  enough,  it  will  affect  the  system  availability. 
Maintenance  personnel  cost  includes  the  total  costs  of  maintenance  personnel  depending 
on  type  of  skill  level. 
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Maintenance  personnel  training  cost  includes  training  of  new  maintenance  technicians. 
On  the  other  hand,  training  of  current  maintenance  personnel  is  necessary  to  keep  up  with 
upgrades  or  changes  in  the  system. 

Measuring  and  test  equipment  cost  includes  testing  and  calibration  of  the  system  and 
adequately  maintaining  the  test  equipment. 

Technical  data  includes  additional  and  new  the  preparation  and  publication  of  system 
installation  and  test  instructions,  operating  and  maintenance  procedures  during  system 
lifetime. 

System  modification  and  upgrading  cost  includes  modernization  and  upgrading  of 
system. 

For  data  generation  estimation  by  analogy  and  estimation  by  expert  judgment 
methods  are  utilized.  In  order  to  calculate  present  value  of  monetary  units  time  value  of 
the  money  is  used.  Since,  this  is  a  decision  situation  involving  money  flow  over  time. 
Interest  rate  is  a  crucial  element  on  time  value  of  money.  The  interest  rate  is  assumed 
14%  due  to  Central  Bank  of  the  Republic  of  Turkey  reports  (www.tcmb.gov.tr,  2011). 
Table  B.2  represents  the  generated  cost  data  for  previously  determined  alternatives. 
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IV.  Results  and  Analysis 


This  chapter  analyzes  the  results  obtained  by  application  of  VFT  process.  These  steps 
consist  of  scoring,  ranking  and  sensitivity  analysis  on  the  developed  model.  Data  for  VFT 
is  gathered  from  the  real  world  for  different  type  of  aircrafts.  Data  for  LCC  is  randomly 
generated  by  using  the  real  world  data  for  the  same  aircrafts;  since,  most  of  the  producer 
companies  refused  to  supply  data.  Assumption  is  made  that  all  data  is  accurate;  because, 
for  the  real  world  situation  producer  companies  are  mandated  to  supply  information. 
Hence,  sensitivity  analysis  is  perfonned  on  objective  weights.  Analysis  results  are 
obtained  by  using  the  Hierarchy  Builder  (Weir,  2008) 

4.1  VFT  Analysis 

4.1.1  Step  6:  Score  and  rank  alternatives  on  each  of  the  value  measures  and  asses 
the  uncertainty  associated  with  the  scores. 

A  score  is  calculated  for  each  alternative  by  summing  up  the  scores  from  each 
value  function  and  the  corresponding  weights  for  each  measure.  The  scores  are  then 
combined  using  equations  lto  give  a  summation  for  each  alternative  and  thus  used  to 
rank  them.  Scores  are  used  only  for  ranking  alternatives;  they  do  not  represent  how  much 
better  one  alternative  is  than  another  one. 

Figure  13  shows  the  ranked  alternatives  in  terms  of  how  well  they  fulfill  each  of 
the  decision  maker’s  objectives  Performance,  Logistics  and  Prestige.  J-10  is  the  best 
alternative  for  Turkish  Stars  aerobatic  team.  Prestige  has  the  biggest  effect  on  scoring 
alternatives;  since,  this  aircraft  is  the  domestic  and  jet  aircraft.  Although  it  does  not  have 
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very  different  scores  on  Logistics  and  Performance ,  it  creates  difference  by  prestige. 
Closest  alternatives  are  J-2  and  J-7.  If  the  production  of  a  domestic  aircraft  will  not  be 
possible  until  the  procurement  period,  then,  it  is  necessary  to  consider  these  two 
alternatives. 


J-10  0.820 
J-2  0.706 
J-7  0.675 
J-6  0.567 
J-9  0.560 
P-4  0.559 
J-4  0.558 
J-3  0.555 
J-5  0.536 
J-l  0.536 
J-8  0.521 
P-3  0.457 
P-2  0.339 
P-1  0.312 


□  Prestige  ■  Logistics  □  Performance 


Figure  83  Overall  Scoring  and  Ranking  of  Alternatives 
By  looking  at  the  value  contributions  of  individual  evaluation  measures,  it 
becomes  clear  which  measures  differentiated  the  alternatives  from  each  other.  Figure  14 
shows  how  well  each  alternative  performed  for  the  17  evaluation  measures.  J-10  has 
respectively  well  performance  for  each  of  the  measures.  It  creates  the  difference  by  origin 
of  the  aircraft,  leasing  right  to  produce  and  life  of  project.  The  main  reason  of  having  a 
better  score  is  being  the  domestic  aircraft  that  creates  advantage  on  Prestige  and  Logistics 
objectives. 
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J-10  0.820 
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J-7  0.675 
J-6  0.567 
J-9  0.560 
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J-4  0.558 
J-3  0.555 
J-5  0.536 
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□  AircraftType 

□  Ease  to  Transport  (International) 

□  Ease  to  Transport  (Domestic) 

□  Air  RefuelingCapability 

□  Leasing  Right  to  Produce 

□  Ease  to  Repair 

□  Maintenance  Personnel  Requirement  □  Aircraft  Prevalence 

□  Low  Skill  Level  □  Medium  Skill  Level 

□  High  Skill  Level 


■  Origin  of  Aircraft 

□  Thrust/Weight  Ratio 

□  Endurance 

□  Life  of  Project 

□  Type  of  Procurement  Agreement 

□  Ease  to  Prepare 


Figure  94  Scoring  and  Ranking  of  Alternatives 
Figure  15  represents  the  contribution  of  each  measure  on  performance  objective. 
The  first  six  alternatives  have  the  advantage  of  air  refueling  capability.  On  the  other  hand, 
all  of  the  propeller  aircrafts  have  the  drawback  of  thrust/weight  ratio.  These  two  issues 
may  prevent  these  alternatives  to  be  one  of  the  candidates. 
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J-3  0.957 


J-5  0.953 
J-7  0.894 
J-10  0.767 
J-2  0.720 
J-4  0.600 
J-l  0.441 
J-6  0.404 
P-2  0.304 
P-1  0.303 
P-4  0.302 
P-3  0.298 
J-9  0.227 
J-8  0.219 


I 


I 


□  Thrust/Weight  Ratio  □  Endurance  □  Air  RefuelingCapability 


Figure  105  Scoring  and  Ranking  of  Alternatives  by  Performance  Objective 
By  looking  at  the  sub-objectives  on  objective  related  to  Logistics  in  Figure  16  and 
Figure  17,  propeller  aircrafts  have  the  best  scores  among  all  other  alternatives.  P-4  has 
the  advantage  of  being  a  domestic  aircraft.  It  is  obvious  that  propeller  aircrafts 
necessitates  less  personnel  and  effort  to  maintain  them.  They  also  need  less  spare  parts 
and  less  transportation  kits,  that  creates  advantage  on  supply  support  and  support 
equipment.  J-10  has  the  advantage  of  being  a  domestic  aircraft  on  supply  support  sub¬ 
objective. 
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Figure  1 16  Overall  Scoring  and  Ranking  of  Alternatives  by  Logistics  Objective 
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□  Ease  to  Transport  (International) 

□  Life  of  Project 

□  Type  of  Procurement  Agreement 

□  Ease  to  Prepare 

□  Aircraft  Prevalence 

□  Medium  Skill  Level 


□  Ease  to  Transport  (Domestic) 

□  Leasing  Right  to  Produce 
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□  Maintenance  Personnel  Requirement 

□  Low  Skill  Level 

□  High  Skill  Level 


Figure  127  Scoring  and  Ranking  of  Alternatives  by  Logistics  Objective 
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Figure  18  represents  the  contribution  of  each  measure  on  prestige  objective.  J-10 
has  the  advantage  of  being  a  domestic  and  jet  aircraft.  Alternatives  J-7  and  J-2  lose  points 
from  their  origins;  but  that  is  not  a  big  loss.  Origin  of  the  aircraft  is  the  main  measure  that 
separates  these  two  alternatives  from  others.  If  we  look  at  Figure  14  that  becomes 
obvious.  Prestige  objective  is  the  greatest  loss  for  the  propeller  aircraft.  Although  their 
performance  is  satisfying  in  terms  of  other  objectives,  the  importance  (weight)  of  prestige 
hurts  the  scores  of  these  alternatives. 


J-10 

1.000 

J-7 

0.871 

J-2 

0.871 

J-9 

0.700 

J-8 

0.700 

J-6 

0.700 

J-4 
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0.700 

J-5 
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J-l 

0.571 

P-4 

0.429 

P-3 

0.300 

P-2 

0.129 

P-1 

0.000 

□  AircraftType  0  Origin  of  Aircraft 


Figure  138  Scoring  and  Ranking  of  Alternatives  by  Prestige  Objective 

4.1.2  Step  7:  Analyze  the  sensitivity  of  the  analysis  to  assumptions  and  consider 
refining  alternatives  to  create  better  alternatives. 

Sensitivity  analysis  is  a  method  of  verifying  that  the  model  is  built  on  proper 
assumptions.  One  of  the  biggest  assumptions  in  the  model  is  that  the  evaluation  measures 
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have  been  given  the  proper  weighting  and  accurately  depict  the  decision  maker’s 
preferences.  Sensitivity  analysis  helps  the  decision-maker  verify  these  weightings  by 
showing  how  the  ranking  of  alternatives  may  change  based  on  variations  in  measure 
weights. 

Graphical  demonstration  is  conducted  to  illustrate  how  each  alternative  will 
receive  more  or  less  value  depending  on  the  weight  of  the  selected  evaluation  measure. 
Sensitivity  analysis  will  be  made  for  objectives  weights  that  are  Performance,  Logistics 
and  Prestige.  Because,  sub-objective  weights  do  not  change  the  results  of  the  current 
model.  On  the  other  hand,  decision  maker  for  this  model  will  be  a  top-level  decision 
maker  that  deals  with  the  major  objectives. 

Sensitivity  Analysis  on  Performance  Objective 

Decision  maker’s  weight  is  originally  0.15  for  Performance  which  is  indicated  by 
the  vertical  line.  Alternatives’  cross  point  on  this  line  indicate  their  respective  rankings. 
Ranking  from  the  top  to  bottom  is  the  ranking  from  the  best  to  worst  alternative.  By 
visual  inspection  of  Figure  19,  it  is  obvious  that  J-10  will  always  be  the  best  choice  if  the 
weight  of  Performance  stays  below  0.6.  Between  0.6  an  0.715  J-7  and  above  0.715  J-3 
will  be  the  best  alternative.  Hence,  if  Performance  gets  more  important  J-7  and  J-3 
becomes  best  alternatives.  In  addition,  J-5  has  close  results  to  J-3.  All  other  alternatives 
will  be  dominated  by  J-10,  J-7  and  J-3.  If  J-10  will  not  be  available  during  the 
procurement  process,  J-2  will  be  the  best  alternative  below  0.28.  Above  that  weight,  J-7 
and  then  J-3  will  be  the  best  alternatives. 
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Figure  19  Sensitivity  Analysis  for  Performance  Objective 
Sensitivity  Analysis  on  Logistics  Objective 

The  next  sensitivity  analysis  is  made  on  the  objective  of  Logistics.  Decision 
maker’s  weight  is  originally  0.35  for  Logistics,  which  is  indicated  by  the  vertical  line.  By 
visual  inspection  of  Figure  20,  it  is  obvious  that  J- 10  will  always  be  the  best  choice  below 
0.675.  Above  that  weight,  P-4  will  be  the  best  alternative.  Other  alternatives  will  always 
be  dominated  these  two  alternatives.  If  J-10  and  P-4  will  not  be  available  during  the 
procurement  process,  J-2  will  be  the  best  alternative  on  the  current  weight.  Between  0 
and  0.2  J-7,  between  0.2  and  0.654  J-2  and  above  0.654  P-3  will  be  the  best  alternatives. 
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Figure  140  Sensitivity  Analysis  for  Logistics  Objective 
Sensitivity  Analysis  on  Prestige  Objective 

Last  sensitivity  analysis  is  conducted  on  the  objective  of  Prestige.  Decision 
maker’s  weight  is  originally  0.5  for  Prestige,  which  is  indicated  by  the  vertical  line  in 
Figure  21.  By  visual  inspection  of  Figure  21,  it  is  obvious  that  J-10  will  always  be  the 
best  choice  if  Prestige  gets  more  important.  If  it  gets  less  important,  P-4  will  be  the  best 
alternative.  This  change  occurs  if  weight  decreases  below  0.08.  If  J-10  and  P-4  will  not 
be  available  during  the  procurement  process,  J-2  will  be  the  best  alternative  on  the 
current  weight.  Between  0  and  0.04  P-1,  between  0.04  and  0.12  P-3  and  above  0.12  J-2 
will  be  the  best  alternatives. 
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Figure  151  Sensitivity  Analysis  for  Prestige  Objective 
Overall  Sensitivity  Comments 

J-10,  J-7  and  J-3  are  the  best  alternatives  due  to  the  sensitivity  analysis  for 
Performance.  J-10  and  P-4  are  the  best  alternatives  by  considering  sensitivity  analysis  for 
Logistics.  J-10  and  P-4  are  the  best  alternatives  if  we  look  at  the  Prestige.  All  other 
alternatives  are  always  dominated  with  other  alternatives.  This  gives  nearly  parallel 
results  to  the  scoring  of  the  alternatives. 

Table  2  illustrates  how  much  the  current  weight  should  be  changed  in  order  to 
switch  another  alternative  from  J-10  including  domestic  alternatives.  In  order  to  replace 
J-10,  Performance  requires  300%  increase  in  its  weight,  Logistics  requires  92.9% 
increase  in  its  weight  and  Prestige  requires  84%  decrease  in  its  weight.  Hence,  we  can 
conclude  that  this  model  has  robust  results  due  to  the  sensitivity  analysis  on  current 
weights  of  main  objectives. 
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Table  2  Summary  of  Sensitivity  Analysis  with  Current  Alternatives 


Objectives 

Current 

Adjusted 

Percent 

Current  Best 

New  Best 

Weight 

Weight 

Change 

Alternative 

Alternative 

Performance 

0.15 

0.6 

300 

J-10 

J-7 

Logistics 

0.35 

0.675 

92.9 

J-10 

P-4 

Prestige 

0.5 

0.08 

-84 

J-10 

P-4 

Without  domestic  aircrafts,  J-2  and  J-7  are  the  best  alternatives  due  to  the 
sensitivity  analysis  for  Performance.  J-2,  J-7  and  P-3  are  the  best  alternatives  by 
considering  sensitivity  analysis  for  Logistics.  J-2,  P-1  and  P-3  are  the  best  alternatives  if 
we  look  at  the  Prestige.  All  other  alternatives  are  always  dominated  with  other 
alternatives. 

Table  3  gives  the  necessary  change  on  current  weight  to  switch  another 
alternative  from  J-2  excluding  domestic  alternatives.  In  order  to  replace  J-2,  Performance 
requires  86.7%  increase  in  its  weight,  Logistics  requires  42.9%  decrease  or  86.9% 
increase  in  its  weight  and  Prestige  requires  76%  decrease  in  its  weight.  In  conclusion, 
this  model  again  gives  robust  results  due  to  the  sensitivity  analysis  on  current  weights  of 
main  objectives  by  excluding  domestic  alternatives. 

Table  3  Summary  of  Sensitivity  Analysis  without  Domestic  Alternatives 


Objectives 

Current 

Adjusted 

Percent 

Current  Best 

New  Best 

Weight 

Weight 

Change 

Alternative 

Alternative 

Performance 

0.15 

0.28 

86.7 

J-2 

J-7 

Logistics 

0.35 

0.2 

-42.9 

J-2 

J-7 

0.654 

86.9 

J-2 

P-3 

Prestige 

0.5 

0.12 

-76 

J-2 

P-3 
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4.2  LCC  Analysis 

LCCA  requires  many  assumptions,  data  collection  and  estimating  relationships 
between  elements.  Before  making  the  final  recommendation,  overall  process,  data  and 
assumptions  should  be  analyzed.  Most  of  the  times  DM  is  interested  in  all-possible 
outcomes  that  can  be  the  result  of  changes  in  current  situation  or  based  on  errors  made. 

There  are  many  areas  of  concern  due  to  error,  uncertainty  and  risk.  Some  decision 
independent  parameters  are  annual  equivalent  cost  data,  present  equivalent  cost  data, 
inflation  rate,  interest  rate,  discount  rate  and  labor  rate  may  affect  the  future  cost.  All 
these  parameters  should  be  part  of  sensitivity  analysis.  On  the  other  hand,  high  cost 
contributors  may  have  important  effects  on  overall  cost.  By  determining  the  high  cost 
contributors,  it  is  possible  to  detect  their  relationships  with  other  factors  that  may  change 
the  result  of  analysis  (Fabrycky,  J.,  &  Blanchard,  1991).  Inclusion  of  other  information 
over  the  time  will  cause  changes;  since,  analysis  is  made  in  a  limited  amount  of  time. 

In  order  to  include  all  uncertainties,  a  high  and  low  level  estimation  can  be  useful. 
By  assuming  DM’s  tolerance  is  a  10%  change  from  the  present  costs,  we  can  define  a 
lower  and  upper  value  for  present  cost  of  each  alternative.  If  past  data  is  available,  it  is 
possible  to  use  statistical  regression  analysis  to  determine  these  upper  and  lower  values. 
Simply  by  increasing  and  decreasing  the  present  cost  of  each  aircraft  by  10%,  lower  and 
upper  estimates  are  found.  Table  B.3  illustrates  results  after  the  calculation. 
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4.3  Cost-Effectiveness  Analysis 

Cost-effectiveness  analysis  deals  especially  with  the  problems  that  the  output 
cannot  be  measured  by  market  prices.  It  is  used  for  the  problems  that  aims  to  maximize 
effectiveness  subject  to  a  resource  constraint  measured  in  monetary  units.  Cost- 
effectiveness  analysis  is  suitable  if  it  is  not  possible  to  evaluate  the  outputs  of  alternatives 
with  a  market  evaluation  and  the  resource  inputs  can  be  evaluated  by  market  prices 
(Goldman,  1967). 

DM  has  two  objectives  that  are  minimizing  cost  and  maximizing  the  value  by  a 
proper  decision.  Although  the  results  determine  fourteen  different  alternatives,  not  all  of 
them  are  the  preferable.  If  alternatives  are  possible  solutions  for  that  decision  problem, 
we  may  define  preferable  alternatives  as  non  inferior  solutions.  A  non  inferior  solution  is 
better  on  all  objectives  than  other  solutions. 

Figure  22  illustrates  the  results  of  LCCA  and  VFT.  Graphically,  the  non  inferior 
solutions  are  the  outer  border  of  the  feasible  solutions  in  the  direction  of  improving 
objectives  (Neufville,  1990).  The  preferred  alternative  should  be  one  of  the  non  inferior 
solutions;  since,  only  the  noninferior  solutions  are  worthwhile. 

Inspection  of  graph  defines  J-10,  J-2,  J-6,  J-9,  P-4,  P-3  and  P-2  as  the  noninferior 
solutions.  In  the  absence  of  alternatives  J-10  and  P-4,  the  noninferior  solutions  will  be  J- 
2,  J-6,  J-9,  P-3  and  P-2.  The  line  between  the  alternatives  approximately  represents  the 
efficient  frontier.  If  there  were  many  alternatives,  it  would  be  necessary  to  construct  a 
trade-off  function  between  value  and  cost.  Yet,  for  this  study,  it  is  not  useful. 
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All  other  alternatives  are  dominated  by  these  noninferior  solutions.  J-10  has  the 
best  performance  on  value  and  P-2  has  the  best  performance  on  cost.  Therefore,  these  two 
alternatives  represent  a  major  alternative  group.  Other  alternatives  lie  between  these  two 
alternatives;  hence,  they  construct  a  compromise  group.  If  DM  has  specific  thresholds, 
some  alternatives  may  be  excluded  from  consideration.  Propeller  aircrafts  have  close 
results.  By  increasing  budget  by  $318000  per  aircraft,  it  is  possible  to  increase  value  by 
0.1  from  P-3  to  P-4.  On  the  other  hand,  $6.6  million  budget  increase  per  aircraft  leads 
0.22  value  increase  from  P-2  to  P-4.  Yet,  this  graph  uses  data  that  is  assumed  true. 


Figure  162  Cost-  Effectiveness  Analysis 

By  inspecting  the  same  graph,  it  will  be  obvious  that  there  is  big  gap  between 
alternatives  J-2  and  P-4  as  illustrated  in  Figure  23.  In  order  to  fill  that  gap,  different 
strategies  may  be  applied  to  the  analysis.  It  is  possible  to  miss  some  other  appropriate 
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alternatives  from  this  analysis.  There  may  be  different  alternatives  that  have  more  value 
for  less  cost.  Hence,  it  can  be  appropriate  to  search  for  new  alternatives.  From  a  different 
point  of  view,  some  of  the  current  alternatives  may  be  taken  to  noninferior  solutions.  For 
instance,  with  different  modifications  enhancing  the  thrust/weight  ratio  for  J-8  we  may 
increase  its  value  with  an  increase  in  cost.  Adding  an  air  refueling  system  to  J-l  may 
carry  that  alternative  to  efficient  frontier. 

Cost  vs.  Value  Chart 
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Figure  173  Gap  between  Cost  and  Value 

Figure  24  illustrates  result  of  LCC  and  VFT  sensitivity  analysis  to  see  the  model 
reflection  to  possible  changes.  Possible  uncertainties  cause  intersections  between  some  of 
the  alternative  areas;  hence,  it  is  not  clear  which  alternative  is  the  preferred  one.  As 
indicated  in  the  Figure  24  where  there  intersections  occur  across  alternatives,  there  is 
uncertainty  as  to  which  system  will  be  a  better  decision  in  terms  of  cost  and  value.  This 
may  cause  risky  decisions.  If  DM  does  not  have  the  insight  of  cost  uncertainty,  it  is  most 
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likely  he  or  she  will  choose  the  alternative  that  has  lowest  cost  estimate  due  to  the  budget 
constraint. 

Value  ranges  are  calculated  by  taking  minimum  and  maximum  limits  for  sub¬ 
objective  weights.  Sensitivity  analysis  is  based  on  the  range  from  zero  to  one.  However, 
this  situation  is  not  very  realistic.  If  an  objective  has  a  zero  weight,  it  is  not  logical  to  put 
it  into  the  hierarchy.  Therefore,  it  is  assumed  that  weight  on  Prestige  may  differ  between 
0.1  and  0.3,  weights  on  Logistics  may  differ  between  0.15  and  0.55  and  weights  on 
Prestige  may  differ  between  0.3  and  0.7. 

Due  to  these  weight  ranges,  alternatives  J-3  and  J-7  may  have  big  changes  on 
value.  It  is  possible  to  have  values  of  0.438  and  0.672  by  spending  $665.2  million  per  J-3 
aircraft.  By  inspecting  values  for  J-7,  it  is  possible  to  make  same  comments.  Propeller 
aircraft  data  produce  the  same  conclusion.  Especially  P-1  may  have  the  value  between 
0.174  and  0.45  that  means  a  significant  uncertainty  for  this  type  of  big  project  decision. 
J-l  has  the  least  uncertainty  in  terms  of  value.  By  inspecting  the  attribute  data  for  J-l,  it  is 
obvious  that  it  has  average  data  for  each  attribute.  Hence,  changing  objective  weights 
does  not  hurt  this  alternative.  Yet,  it  does  not  produce  value  as  much  as  the  other 
non  inferior  solutions  among  jet  aircrafts. 

As  stated  before,  intersections  between  alternative  areas  are  another  issue  to 
consider.  All  of  the  dominated  solutions  from  previous  table  will  be  dominated  most  of 
the  time.  Yet,  it  is  not  obvious  which  alternative  is  better  than  other  one  at  some  points. 
By  calculating  the  percentage  of  intersection  area  among  the  total  area  of  intersecting 
alternatives,  it  is  possible  to  obtain  the  probability  of  considering  both  alternatives.  If  DM 
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needs  make  a  choice  between  J-2  and  J-10,  8.18  %  of  the  time  he  or  she  will  be 
indifferent  between  these  two  alternatives.  If  these  aircrafts  are  J-l  and  J-6,  then  the 
percentage  will  be  22.23  %  that  implies  more  uncertainty.  On  the  other  hand,  DM  will  be 
indifferent  between  J-l  and  J-8  9.22%  of  the  time  and  For  J-8  and  J-9  20.1%  of  the  time. 
For  the  propeller  aircrafts,  30.24  %  of  the  time  decision  maker  may  be  indifferent 
between  P-3  and  P-4. 

On  the  other  hand,  there  is  a  big  difference  between  most  of  the  jet  aircrafts  and 
propeller  aircrafts  in  terms  of  value  and  cost.  By  paying  more,  DM  increase  the  value  of 
the  alternative.  In  addition,  propeller  aircrafts  has  a  big  range  on  value  sensitivity  in 
comparison  to  jet  alternatives.  Therefore,  it  is  possible  to  conclude  that  by  paying  more 
DM  increases  the  value  and  decreases  the  risk. 

Adding  thresholds  may  be  useful.  By  that  way,  it  is  possible  to  exclude  the 
alternatives  that  do  not  satisfy  the  DM’s  lower  limit  for  value  and  upper  limit  for  cost. 
Exclusion  of  these  alternatives  makes  the  analysis  easier  and  other  alternatives  more 
visible.  There  are  only  fourteen  alternatives  for  this  research;  so,  it  is  not  too  challenging 
to  observe  the  alternatives.  Yet,  different  studies  with  greater  number  of  alternatives  may 
be  more  difficult  to  draw  conclusions  without  these  thresholds. 
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Cost-Effectiveness  Chart 
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Figure  184  Cost-Effectiveness  Analysis  with  LCC  and  VFT  Sensitivity  Analysis 
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IlfH  ItHHH 


V.  Summary  and  Conclusion 


This  chapter  provides  a  brief  description  of  this  research,  answers  the  questions 
stated  in  Chapter  1 ,  states  conclusions  that  are  obtained  from  the  research  and  suggestions 
for  future  work. 

5.1  Summary  of  the  Research 

In  Chapter  1  problem  definition,  research  questions,  research  approach  and 
assumptions  are  stated. 

Chapter  2  includes  techniques  and  related  studies  for  these  techniques.  At  first,  a 
brief  description  of  Turkish  Procurement  Law  illustrates  procedures  used  for 
governmental  and  military  procurements  to  look  for  possible  limitations  of  the  law  for 
techniques.  VFT  technique  is  used  to  create  different  alternatives  that  have  appropriate 
attributes  for  DM  and  stakeholders  of  that  decision  problem.  LCCA  is  conducted  to 
estimate  and  analyze  the  present  cost  of  each  alternative.  Steps  are  briefly  described  and 
different  infonnation  sources  are  recommended. 

Chapter  3  presents  methodology  and  implementation  of  VFT  and  LCCA.  First,  a 
value  hierarchy  is  created  to  define  desired  attributes  of  alternatives.  Then  DM  and  SMEs 
stated  SDVF  for  each  measure  of  the  hierarchy.  Alternative  identification  is  made  due  to 
the  measures  of  the  hierarchy.  After  defining  weights,  the  additive  value  model  is  used  to 
find  the  overall  score  and  ranking  of  each  alternative.  LCCA  steps  describe  the  definition 
and  CBS  of  the  current  problem.  Data  for  VFT  and  LCCA  are  generated  by  using 
different  real  world  aircraft  data. 
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Analysis  phase  is  presented  in  Chapter  4.  VFT  sensitivity  analysis  is  made  on 
main  objectives.  Since  the  most  important  assumption  is  the  proper  weight  data.  For 
LCCA  sensitivity,  many  issues  should  be  taken  into  account.  By  just  simply  assuming 
only  a  specific  amount  of  deviation  is  acceptable,  cost  range  for  each  alternative  is 
calculated.  In  order  to  combine  VFT  and  LCCA  results,  cost-effect6ive  analysis  is 
conducted.  All  these  analyses  are  made  to  conclude  if  the  model  is  robust. 

5.2  Conclusions 

Difficulty  of  the  decision-making  process  stems  from  its  complexity,  uncertainty, 
multiple  objectives,  and  integration  of  different  data  types.  In  addition,  results  of  that 
process  affect  a  large  number  of  people,  a  large  amount  of  resources  and  a  long  period. 
On  the  other  hand,  all  of  the  DM  wants  to  make  a  decision  that  they  may  achieve  their 
objectives.  Hence,  a  systematic  technique  is  needed  to  analyze  this  process.  VFT  in 
conjunction  with  MODA  provides  this  systematic  approach. 

The  purpose  of  this  research  effort  is  to  evaluate  the  alternatives  that  maximize 
cost  effectiveness  in  the  decision  process  of  TURAF  by  using  a  VFT  approach  with 
LCCA.  Eight  steps  of  VFT  that  are  stated  by  Kirkwood  are  used  to  develop  a  robust 
model  for  TURAF  decision  problem.  The  robustness  of  the  model  is  checked  by 
sensitivity  analysis.  For  this  research,  DM  and  SMEs  are  satisfied  by  the  results  for 
weighting  and  weights  reflect  their  preference  ranking.  Hence,  results  are  verified. 

In  addition  to  VFT,  LCCA  steps  are  utilized  to  calculate  LCC  of  each  alternative. 
LCCA  gives  a  different  perspective  for  DM.  Since,  LCCA  detennines  the  cost 
effectiveness  of  the  alternatives  by  analyzing  long  tenn  costs  by  using  time-value  of 
money.  If  only  the  VFT  results  are  considered,  there  will  be  less  than  three  alternatives 
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that  the  DM  should  consider.  Yet,  combination  of  these  two  techniques  provides  a  better 
insight.  Because  of  the  fact  that  circumstances  may  change  over  the  time,  it  may  be 
necessary  to  make  tradeoff  between  cost  and  value  depending  on  the  budget. 
Combination  of  VFT  and  LCCA  shows  other  alternatives  that  may  be  considered 
regarding  that  tradeoff.  This  methodology  may  be  used  for  similar  decision  processes. 

This  research  shows  that  VFT  with  LCCA  is  a  suitable  methodology  for  analyzing 
different  alternatives.  However,  the  results  are  only  a  recommendation;  every  DM  has  to 
base  their  final  decision  on  a  variety  of  factors.  The  value  model  aids  the  decision  making 
process  by  identifying  the  core  values  and  measures  that  should  be  considered  while 
selecting  alternatives.  Too  many  factors  have  varying  impacts  on  this  decision-making 
process. 

5.3  Strengths 

Decision  Analysis  process  may  be  a  complicated  process  for  most  of  the  decision 
makers.  Most  of  the  points  can  be  misunderstood  without  the  presence  of  decision 
analysis  background.  VFT  overcomes  all  of  these  drawbacks  of  DA.  By  defining  the 
attributes  and  their  values,  alternatives  are  selected  based  on  the  DM  considerations  by 
reducing  biases  of  choosing  alternatives  directly  from  the  real  world.  Therefore,  using 
this  technique  allows  an  objective  analysis  for  the  DM  in  comparison  to  other  techniques 
in  the  literature. 

In  addition,  DM  does  not  have  to  know  decision  analysis  issues;  and  VFT  does 
not  necessitate  the  explanation  of  tenns  in  the  DA  terminology.  That  makes 
communication  between  analyst  and  DM  easy  and  time  efficient.  DM  and  SMEs 
contribute  decision-making  process  from  different  standpoints  and  makes  the  analysis 
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interactive.  This  participation  makes  the  results  of  the  research  easy  to  explain  and  easy 
to  implement  to  similar  problems.  Hence,  the  model  provides  an  objective  and  repeatable 
process.  Combination  of  VFT  and  LCCA  present  a  good  visual  illustration  of  alternatives 
besides  scoring  and  ranking. 

5.4  Limitations 

This  research  has  some  limitations.  First,  data  for  both  aircraft  attributes  and  costs 
should  be  based  on  the  prospectively  collected  data  from  the  manufacturers.  The  data 
limitation  for  this  study  stems  from  the  unwillingness  of  the  manufacturers  to  give  the 
available  data.  Thus,  for  this  study,  data  used  comes  from  a  variety  of  diverse  sources  and 
data  generation.  On  the  other  hand,  collecting  suitable  data  for  an  appropriate  analysis 
necessitates  a  long  amount  of  time. 

The  results  of  the  sensitivity  analysis  provide  insight  into  which  aircrafts  were 
most  compatible  with  different  weighting  values.  However,  this  analysis  is  among  the 
limited  amount  of  alternatives.  Depending  on  the  different  circumstances,  different  types 
of  alternatives  should  be  considered. 

The  other  limitation  is  the  deferrable  weights  and  measures  to  score  the 
alternatives.  Since  these  depends  on  the  current  DM,  SME  and  leading  of  analyst. 
Weighting  can  make  value  model  very  sensitive  due  to  given  weight  by  decision  maker. 
The  other  limitation  of  the  model  is  bias  introduced  by  a  single  decision  maker  after  the 
detennination  of  alternatives.  That  bias  may  be  on  the  origin  of  the  aircraft  or  similar 
objectives.  It  may  be  reduced  by  using  different  procedures;  nevertheless,  this  bias  may 
not  be  removed  from  the  current  analysis. 
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5.5  Future  Work 


For  future  work  on  the  same  problem  or  similar  problems,  the  following  research  can 
be  done; 

•  Cost  drivers  of  the  model  can  be  defined  to  reduce  and  analyze  the  overall  cost  of 
each  aircraft. 

•  Optimization  procedures  can  be  applied  to  LCCA  after  determining  the  cost 
drivers. 

•  Sensitivity  analysis  can  be  conducted  on  each  parameter  that  affects  cost. 

•  The  same  model  can  be  utilized  with  the  real  world  data. 

•  Simulation  techniques  can  be  used  to  analyze  the  long-term  perfonnance  of  the 
data  and  the  model  and  to  generate  data  to  validate  and  verify  the  theoretical 
model. 

•  Instead  of  single  representative  of  stakeholders,  group  stakeholders  can  be 
considered. 
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Appendix  A.  Value  Functions 


The  minimum  score  (90)  has  value  of  zero,  the  score  180  has  value  of  0.8  and  the  score 
of  350  has  value  of  one. 


Figure  A.  1  Value  Function  of  Endurance 

The  most  preferred  capability  is  having  air  refueling;  hence,  if  an  aircraft  has  air  refueling 
capability  the  score  will  be  one.  Otherwise,  the  score  will  be  zero. 


Figure  A.  2  Value  Function  of  Air  Refueling  Capability 
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The  minimum  score  (0.85)  has  value  of  zero,  the  score  1  has  value  of  0.8  and  the  score  of 


1 .2  has  value  of  one. 


Figure  A.  3  Value  Function  of  Thrust/  Weight  Ratio 
The  minimum  score  (75)  has  value  of  one,  the  score  180  has  value  of  0.8,  the  score  230 
has  value  of  0.3  and  the  score  of  300  has  value  of  zero. 


Figure  A.  4  Value  Function  of  Ease  to  Repair 
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The  minimum  score  (120)  has  value  of  one,  the  score  400  has  value  of  0.8,  the  score  530 


has  value  of  0.3  and  the  score  of  1 100  has  value  of  zero. 


Figure  A.  5  Value  Function  of  Ease  to  Prepare 
The  minimum  score  (30)  has  value  of  one,  the  score  800  has  value  of  0.8,  the  score  110 
has  value  of  0.3  and  the  score  of  230  has  value  of  zero. 


Figure  A.  6  Value  Function  of  Maintenance  Personnel  Requirement 
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The  minimum  score  (0.1)  has  value  of  zero,  the  score  0.3  has  value  of  0.8  and  the  score 


of  0.4  has  value  of  one. 


Figure  A.  7  Value  Function  of  Low  Skill  Level 
The  minimum  score  (0.1)  has  value  of  zero,  the  score  0.4  has  value  of  0.8  and  the  score 
of  0.6  has  value  of  one. 


Figure  A.  8  Value  Function  of  Medium  Skill  Level 
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The  minimum  score  (0.2)  has  value  of  one,  the  score  0.3  has  value  of  0.8  and  the  score  of 


0.7  has  value  of  zero. 


Figure  A.  9  Value  Function  of  High  Skill  Level 
The  minimum  score  (150)  has  value  of  zero,  the  score  4000  has  value  of  0.8  and  the  score 
of  4500  has  value  of  one. 


Figure  A.  10  Value  Function  of  Aircraft  Prevalence 
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The  minimum  score  (0)  has  value  of  zero,  the  score  0.5  has  value  of  0.3,  the  score  0.9  has 


value  of  0.8  and  the  score  of  1  has  value  of  one. 


Figure  A.  1 1  Value  Function  of  Leasing  Right  to  Produce 
The  most  preferred  one  is  making  an  agreement  with  domestic  suppliers;  hence,  if  an 
aircraft  has  domestic  procurement  agreement  the  score  will  be  one.  Otherwise,  the  score 
will  be  0.85  for  PBL,  0.7  for  FMS,  0.3  for  NAMSA  and  0  for  MOA. 


Figure  A.  12  Value  Function  of  Type  of  Procurement  Agreement 
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The  minimum  score  (0)  has  value  of  zero,  the  score  30  has  value  of  0.8  and  the  score  of 


40  has  value  of  one. 


Figure  A.  13  Value  Function  of  Life  of  Project 


The  minimum  score  (0)  has  value  of  one,  the  score  10  has  value  of  0.8,  the  score  12  has 
value  of  0.3  and  the  score  of  20  has  value  of  zero. 


Figure  A.  14  Value  Function  of  Ease  to  Transport  (International) 
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The  minimum  score  (0)  has  value  of  one,  the  score  8  has  value  of  0.8,  the  score  9  has 


value  of  0.3  and  the  score  of  14  has  value  of  zero. 


Figure  A.  15  Value  Function  of  Ease  to  Transport  (Domestic) 

The  most  preferred  one  is  cat-1  that  is  the  domestic  aircraft;  hence,  if  an  aircraft  is 
domestic  the  score  will  be  one.  Otherwise,  the  score  will  be  0.7  for  cat-2,  0.3  for  FMS, 
0.3  for  cat-3  and  0  for  cat-4. 


Figure  A.  16  Value  Function  of  Origin  of  Aircraft 
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The  most  preferred  aircraft  is  the  jet  aircraft;  hence,  if  an  aircraft  is  a  jet  aircraft  the  score 
will  be  one.  Otherwise,  the  score  will  be  zero. 


Figure  A.  17  Value  Function  of  Aircraft  Type 
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Appendix  B.  Aircraft  Attribute  and  Cost  Data 
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INTRODUCTION 


•The  mission  of  an  aerobatic  team  is  to  enhance 
government  recruiting,  and  credibly  represent  an 
Air  Force. 


•Turkish  Air  Force  Command  plans  to  replace  the 
current  aerobatic  aircraft,  which  have  completed 
their  lifetime. 


1st  Lt.  Emel  Bertgoz(TurAF) 
Advisor:  Dr.  Jeffers’  D.  Weir 
Reader:  Lt.Col.  Doral  E.  Sandlin,  PhD 

Department  of  Operational  Sciences  (ENS) 
Air  Force  Institute  of  Technology 


Value  Model 


CONCLUSION 


•The  purpose  of  this  research  effort  is  to  evaluate 
the  alternatives  that  minimize  cost  and  maximize 
value. 


•The  value  model  reflects  the  main  concerns  of  the 
DM.  stakeholders  and  SMEs. 

•VFT  provides  a  systematic  approach  and  it 
overcomes  the  drawbacks  of  DA. 


•A  decision  making  process  will  be  used  by 
TurAF  to  select  the  best  cost  effective  alternative 
while  taking  into  account  the  values  of  the 
decision  maker  and  stakeholders. 


•The  purpose  of  this  research  is  to  lessen  the 
potential  human  errors  by  using  decision 
analysis  and  cost  analysis  techniques. 


•This  research  uses  Value-  Focused  Thinking  in 
conjunction  with  Multi-Objective  Decision 
Analysis  and  Cost  Analysis. 

•It  creates  a  decision-making  model  that  allows 
decision  makers  to  interact  with  analysts  by 
specifying  his/her  objectives,  values  and 
preferences. 


•Cost  analysis  process  determines  cost  effective 
alternatives  are. 


■This  methodology  may  be  used  for  similar  decision 
processes. 

•This  technique  allows  for  an  objective  analysis  for 
the  DM  . 

■Combination  of  VFT  and  LCCA  present  a  good 
visual  illustration  of  alternatives  . 
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•Cost  drivers  of  the  model  can  be  defined. 
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•Optimization  procedures  can  be  applied  to 
Life  Cycle  Cost  Analysis. 

•Sensitivity  analysis  can  be  conducted  on 
each  parameter  that  affects  cost. 

•Real  world  data  can  be  utilized. 


•Simulation  techniques  can  be  used  to  analyze 
the  long-term  performance. 


•Group  stakeholders  can  be  considered. 
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